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FOREWORD 


Thi9  bibliography  has  been  prepared  as  a  convenient  reference  to  recent 
research  publications  in  the  command  and  control  area  by  the  U.S.  Army  Re¬ 
search  institute  (AR1) .  Research  in  this  domain  currently  is  conducted  pri¬ 
marily  by  the  Human  Factors  Technical  Area,  Alexandria,  and  the  Fort  Leaven¬ 
worth  Field  Unit.  Publications  have  been  divided  into  a  set  of  convenient 
areas  of  research  and  are  listed  in  chronological  order  within  each  area. 

In  early  1979,  the  variety  of  documentation  types  was  reduced  to  the 
Research  Report,  Technical  Report,  Research  Note,  and  Working  Paper.  Sev¬ 
eral  earlier  types  of  reports  appear  in  this  bibliography.  The  various  ARi 
documentation  types  are  distinguished  by  the  primary  intent  of  the  report, 
as  follows: 

Research  Reports  describe  completed  research  studies  or  programs  which 
contribute  directly  to  the  solution  of  Army  human  factors  problems.  Reports 
are  distributed  to  their  sponsors,  research  and  operational  facilities  in 
the  Department  of  Defense  and  to  interested  civilian  agencies  and  educational 
institutions.  Research  Reports  are  deposited  in  the  Defense  Technical  Infor¬ 
mation  Center  (DTIC)  and  the  National  Technical  Information  Services  (NTIS) 
for  public  retrieval.  (Called  Technical  Research  Reports  1941-1973.) 

Technical  Reports  present  technical  information  concerning  research 
methodology,  basic  psychological  knowledge  or  concept  feasibility,  growing 
out  of  research  programs.  They  are  primarily  of  interest  to  technically 
trained  research  workers  in  tho  Department  of  Defense  and  in  other  govern¬ 
mental  research  agencies.  They  are  distributed  chiefly  to  such  agencies,  to 
education  institutions,  and  to  DTIC  and  NTIS.  Distribution  to  relevant  mili¬ 
tary  organizations  is  also  made  for  their  use  in  advanced  planning  and  for 
application  of  serendipitous  findings  to  more  proximate  problems.  (Called 
Technical  Research  Notes  1952-1973  and  Technical  Papers  1973-1979.) 

Research  Notes  report  contract  research  of  immediate  interest  to  an  im¬ 
portant  but  very  small  audience.  Copies  are  not  printed  for  formal  ARI  dis¬ 
tribution,  but  may  be  obtained  from  DTIC  and  NTIS. 

Working  Papers,  for  limited  distribution  only,  present  technical  infor¬ 
mation  of  interest  within  ARI  or  as  interim  or  partial  report  to  specific 
agencies  supported  by  ARI.  Distribution  is  generally  restricted  to  ARI  and 
sponsor  personnel. 

Research  Problem  Reviews,  1958-1979,  were  special  reports  to  military 
management,  generally  prepared  in  response  to  questions  raised  by  operating 
agencies  when  early  answers  were  needed.  Distribution  vis  primarily  to 
operating  agencies  with  a  direct  interest  in  the  content,  but  copies  of  many 
may  now  be  obtained  through  DTIC  and  NTIS.  Discontinued  in  1979. 

Research  Memorandums ,  1951-1979,  were  informal  reports  on  technical 
research  problems  and  methodological  developments  relating  primarily  to 
in-house  technical  operations.  Distribution  of  these  publications  was 
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generally  limited  to  personnel  engaged  in  research  for  ARI,  but  copies  of 
many  may  now  be  obtained  from  DTIC  and  NTIS.  Discontinued  in  1979. 

Technical  Reports  (TR-A,  TR-B,  and  TR-TH  series),  1975-1979,  were  pre¬ 
pared  by  a  contractor  or  grantee  on  contract  research  developed  and  tech¬ 
nically  monitored  by  ARI .  Distribution  could  be  that  of  a  Research  Report 
(A  and  TH  series)  or  of  a  Research  Problem  Review  (B  series) .  Discontinued 
in  1979. 

preliminary  reports  which  have  not  been  revised  and  edited  for  publi¬ 
cation  because  of  the  recency  of  the  research  have  been  identified  as  Draft. 
Additionally,  reports  which  were  not  formally  published  either  because  of 
changing  requirements,  small  specialized  audiences,  or  urgency  of  other  ef¬ 
forts  also  have  been  identified  as  Draft.  Draft  reports  are  primarily  of 
immediate  interest  to  sponsors  as  well  as  to  future  research  efforts,  and 
their  distribution  is  generally  limited  to  personnel  engaged  in  research 
for  ARI  and  to  direct  sponsors  of  that  research. 

Letter  reports  and  other  results  of  technical  advisory  services  to  spon¬ 
sors  have  not  been  included  because  of  their  specific  nature,  inclusion  of 
proprietary  or  privileged  information,  or  tailoring  for  a  limited  audience. 
Distribution  of  such  reports  is  usually  restricted  to  the  sponsor. 

Qualified  agencies  and  individuals  not  on  initial  ARI  distribution  may 
obtain  copies  of  reports  in  print  from  ARI  as  long  as  stocks  last.  Refer¬ 
ence  copies  may  also  be  available  on  a  limited  loan  basis.  When  the  ARI 
initial  stock  has  been  exhausted,  reports  may  be  purchased  from  the  follow¬ 
ing  sources,  using  the  AD  number  listed  with  each  citation: 

Department  of  Defense  agencies  and  their  contractors: 

Defense  Technical  Information  Center 
Cameron  Station 
Alexandria,  Virginia  22314 

Other  government  agencies  and  qualified  individuals: 

National  Technical  Information  Service 
Department  of  Commerce 
Springfield,  Virginia  22151 
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General 


Rinqel ,  S.  J. ,  Baker,  J.  D.,  strub ,  M.  H. ,  and  Kinsinyar,  L.  L.  Human  fac¬ 
tors  research  in  command  information  processing  systems — Summary  of 
recent  studies.  Technical  Research  Report  1158,  July  1969.  (AD  694  347) 

The  present  report  provides  a  synopsis  of  research  accomplished  from 
March  1966  to  July  1969  in  close  liaison  with  the  Command  Systems  Field 
Branch  established  in  USAFEUR  in  September  1967.  It  includes  principal  find¬ 
ings  and  their  application  to  the  enhancement  of  human  performance  in  the 
Seventh  Army  Tactical  Operations  System  (TOS)  and  facilitation  o.c  man-machine 
interaction  in  relation  to  total  system  effectiveness.  Studies  completed 
prior  to  this  period  are  described  in  BESRL  Technical  Research  'Report  1145, 
"Human  Factors  Research  in  Command  Information  Processing  Systems , "  March 
1966.  \ 


Baker,  J.  D.  Acorns  in  flower  pots/psychologists  in  the  field.  Report  of 
the  Sixteenth  Annual  Army  Human  Factors  Research  and  Development  Con¬ 
ference.  U.S.  Army  Air  Defense  Center  and  Fort  Bliss,  TX,  20-22  October 
1970,  pp.  22-47.  (Available  from  American  Psychological  Association, 
Journal  Supplement  Abstract  Service,  MS  No.  191.) 

i 

This  paper  addresses  itself  to  the  problem  of  Establishing  a  linkage 
between  laboratory  experimentation  and  applied  research  and  development.  It 
describes  a  situation  wherein  a  laboratory- linked  field-unit  approach  was 
taken  to  bridge  the  gap.  The  vehicle  for  the  field  '■  research  portion  of  the 
effort  was  the  United  States  Army's  Tactical  Operations  System  (TOS) .  A 
general  description  and  broad  overview  of  the  system's  operations  is  in¬ 
cluded.  A  common  framework  for  both  the  laboratory  and  field-unit  research 
was  developed  and  used.  Some  representative  studies  that  were  conducted  in 
the  field  are  described. 

Discussion  centers  on  the  use  of  human  factors  research  to  improve  sys¬ 
tem  design  and  the  problems  of  introducing  laboratory  findings  into  real- 
world  operations. 


Baker,  J.  D,  Human  performance  research  in  military  information  systems: 
Quo  fuimus  et  quo  vadimus?  In  Walraven,  )?.  L.  and  Bernotat,  R.  (Ed.). 
Proceedings  of  the  Twelfth  Defense  Research  Group  Seminar  on  the  Opti¬ 
mum  Balance  Between  Man  and  Machine  in  Man-Machine  Systems.  Utrecht , 
The  Netherlands,  NATO  Report  DS/DR(73) 375 ,  May  1973. 

The  1969  NATO  Defense  Research  Group  (DRG)  Seminar  on  the  Design  of 
Equipment  for  Effective  Utilization  (held  at  Mondello,  Italy)  had  identi¬ 
fied  a  number  of  recurring  key  problems  in  man-machine  design.  Among  these 
were  (a)  how  to  get  the  results  of  research  being  done  applied:  (b)  motiva¬ 
tion  as  a  variable,  and  the  absence  of  this  concept  in  the  human  factors 
field?  (c)  the  need  to  restructure  design  procedures  to  include  the  user? 

(d)  the  failure  to  consider  the  dynamics  of  interpersonal  behavior  in  com¬ 
plex  man-machine  systems?  and  (e)  the  question  of  identifying  and  defining 
relevant  criteria. 
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This  paper  deac ribas  some  research  which  addressed  these  problem  areas. 
It  attempts  to  identify  sane  critical  aspects  of  these  questions  which  remain 
unanswered,  and  it  suggests  what  some  of  the  future  developments  in  man- 
machine  systems  are  likely  to  be. 


Andrews,  R.  S.  Human  processes  in  battlefield  information  systems.  In 
Thrall,  R.  M. ,  Tsokos,  C.  P.,  and  Turner,  J.  C.  (Eds.),  Proceedings 
of  the  Workshop  on  Decision  Information  for  Tactical  Command  and  Con¬ 
trol.  Airlie  House,  Airlie,  VA,  22-25  September  1976.  Houston,  TX» 
Robert  M.  Thrall  and  Associates,  1977. 

The  vehicles  for  decision  information  for  Army  tactical  command  and 
control  are  often  identified  as  battlefield  information  systems.  For  the 
foreseeable  future,  human  processes  will  be  an  integral  part  of  such  systems 
and  must  be  attended  to  as  explicitly  as  the  computer  technology  if  signifi¬ 
cant  advances  in  system  capability  are  to  be  achieved.  This  concept  and  its 
implications  are  developed  in  this  paper.  First,  an  attempt  is  made  to  de¬ 
fine  the  Army  problem.  Then  some  of  the  relevant  variables  are  discussed 
with  respect  to  how  their  effects  might  be  measured,  whether  solutions  are 
intuitively  obvious,  and,  finally,  how  advanced  computer  technology  holds 
both  promises  and  pitfalls  in  aiding  the  decision  process. 


Bonder,  S.,  Cherry,  w.  P. ,  and  Farrell,  R.  L.  A  framework  for  identifying 
human  research  needs  in  command  and  control.  Research  Note  80-5, 

March  1980.  (AD  A082  Oil) 

This  effort  developed  a  framework  to  assist  in  identifying  human  factors 
research  areas  in  command  and  control  (HF/c2)  which  are  of  importance  to  the 
Army.  It  is  a  systems -oriented  framework  that  considers  HF/C2  activities 
needed  to  assist  in  developing  Army  systems  and  NH/C2  knowledge  required/ 
available  to  generate  a  menu  of  potential  HF/C2  research  needs  by  type  of 
research  (Application  Demonstration,  Applied  Research,  and  Exploratory  Re¬ 
search)  .  The  framework  also  provides  information  as  to  when  the  research 
output  will  be  of  first  use  to  the  Army. 
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Staff  Organization  and  Operations 
Decision  Support 
Decision  Support:  Graphic  Aids 
Information  Requirements 
Information  Management 
Intelligence  Information  Processing 
Intelligence  Data  Evaluation 
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Staff  Organization  and  Operations 


Staff  Organization  and  Operations 

Modisette,  B.  R. ,  Michel,  R.  R. ,  and  Stevens,  G.  W.  Initial  strategies  for 
the  Tactical  Operations  System  (TOS)  support  of  the  command  and  control 
process.  Volume  Is  Overview  of  TOS  operations.  Technical  Report 
TR-78-A15 ,  June  1978.  (AD  AC58  324) 

The  impact  of  the  Tactical  Operations  System  (TOS)  on  the  procedures, 
personnel,  and  skill  requirements  for  affected  staff  elements  at  Army  division, 
brigade,  and  battalion  command  posts  is  discussed.  Changes  to  the  current 
TOS  design  that  would  enhance  its  usefulness  to  staff  users  are  identified 
and  defined.  The  results  of  this  research  provide  basic  source  material 
useful  for  determining  TOS  training  requirements  and  providing  the  ration¬ 
ale  for  detailed  investigation  of  the  feasibility  and  utility  of  proposed 
system  enhancements. 


Modisette,  B.  R. ,  Michel,  R.  R. ,  and  Stevens,  G.  W.  Initial  strategies  for 
the  Tactical  Operations  System  (TOS)  support  of  the  command  and  control 
process.  Volume  2s  Description  of  TOS  functions  for  division  ele¬ 
ments.  Technical  Report  TR-78-A16,  June  1978.  (AD  A061  155) 

This  report  provides  an  element-by-element  discussion  of  the  TOS  func¬ 
tions  described  in  Volume  1  of  this  series. 


Modisette,  B.  R. ,  M.i.chel,  R.  R. ,  and  Stevens,  G.  W.  Initial  strategies  for 
the  Tactical  Operations  System  (TOS)  support  of  the  command  and  control 
process.  Volume  3:  Description  of  TOS  functions  at  brigade  and  bat¬ 
talion.  Technical  Report  TR-78-A17,  June  1978.  (AD  A061  103) 

This  report  provides  a  discussion  of  TOS  functions  for  brigade  and 
battalion  command  posts,  paralleling  Volume  2  of  this  series. 
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Erickson,  J.  M. ,  and  Levit,  R.  A.  Development  and  application  of  a  decision 
aid  for  tactical  control  of  battlefield  operations:  Bibliographic  sort 
of  the  decision-aiding  literature.  Research  Memorandum  73-3,  December 
‘1973.  (AD  A082  950) 


A  bibliography  on  decision  aiding  and  decisionmaking  in  man/computer 
systems  was  selected  primarily  from  post-1960  research  literature.  Selec¬ 
tions  met  at  least  one  of  the  following  criteria:  (a)  contained  a  working 
decision  aid  in  a  man/computer  decisionmaking  environment;  (b)  contained  a 
model  of  decisionmaking  from  which  an  aid  could  be  derived;  (c)  contained 
useful  information  on  tactical  information  processing;  and  (4)  contained 
useful  information  on  methodological  aspects  of  decision  aiding  in  man/ 
computer  contexts.  The  literature  is  presented  in  four  ways:  alphabeti¬ 
cally  by  first  author,  alphabetically  by  author  within  year  of  publication, 
alphabetically  by  keyword,  and  alphabetically  within  each  of  17 
classifications . 


Levit,  R.  A.,  Erickson,  J.  M. ,  and  Heaton,  B.  J.  Decision  style  measurement 
and  decision  support  software  specifications.  Unpublished  Technical 
Report ,  March  1977. 

This  report  includes  the  computer  printouts  of  che  software  specifica¬ 
tions  for  the  decision  aid  discussed  in  Technical  Reports  TR-77-A2  and 
TR-77-A3.  The  report  is  available  for  inspection  at  ARI  headquarters  in 
Alexandria,  Virginia. 


Levit,  R.  A.,  Alden,  D.  G. ,  Erickson,  J.  M. ,  and  Heaton,  B.  J.  Development 
and  application  of  a  decision  aid  for  tactical  control  of  battlefield 


operations:  A  conceptual  structure  for  decision  support  in  tactical 
Operations  Systems.  Technical  Report  TR-77-A2,  March  1977.  (AD  A040  606) 

A  by-product  of  future  automated  tactical  systems  is  proliferation  of 
data  to  be  organized  and  analyzed.  To  help  cope  with  this  problem  and  bring 
about  an  efficient  man-computer  interface,  an  evaluation  of  the  decision- 
aiding  literature  was  undertaken.  Special  emphasis  was  placed  on  those  aids 
which  might  be  useful  in  tactical  systems  and  could  be  implemented  in  ARI's 
laboratory  simulation  of  an  automated  tactical  operations  center. 


Levit,  R.  A.,  Alden,  D.  G.,  Erickson,  J.  M. ,  and  Heaton,  B.  J.  Development 
and  application  of  a  decision  aid  for  tactical  control  of  battlefield 
operations:  A  preliminary  evaluation  of  a  decision  support  complex 
in  S1MTOS.  Technical  Report  TR-77-A3,  March  1977.  (AD  A040  563) 


The  present  research  provided  a  preliminary  evaluation  of  a  decision 
support  complex  designed  for  use  in  a  simulated  tactical  operations  system 
(S1MT0S) . 

Ten  participants  were  divided  into  two  groups.  One  group  used  the  de¬ 
cision  aid  throughout  the  simulated  defensive  scenario  while  the  other  group 
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did  not.  Both  groups  participated  in  the  same  G3  planning  and  combat 
activities. 

The  general  results  showed  that  the  concept  of  a  decision  support  com¬ 
plex  is  a  sound  one.  ARI  is  continuing  to  utilize  the  decision  support 
complex  in  further  studies  of  tactical  decisionmaking. 


Levit,  R.  A.,  Heaton,  B.  J.,  and  Alden,  D.  G.  Development  and  application 
of  decision  aids  for  tactical  control  of  battlefield  operations:  De¬ 
cision  support  in  a  Simulated  Tactical  Operations  System  (SIMTOS) . 
Technical  Report  77-A13,  December  1977.  (AD  A050  889) 

In  an  experiment  designed  to  evaluate  the  effectiveness  of  an  automated 
decision  support  system  on  tactical  information  processing  and  decisionmak¬ 
ing  performance,  25  experienced  Army  officers  played  the  role  of  Division 
G3  in  the  Simulated  Tactical  Operations  System  (SIMTOS) .  Results  of  the 
evaluation  demonstrated  that  the  concept  of  decision  support  is  sound,  al¬ 
though  specific  measures  of  tactical  performance  were  insensitive  to  player 
actions.  The  decision  support  complex  increased  players'  ability  to  inter¬ 
act  effectively  with  SIMTOS  and  increased  their  satisfaction  with  the  system. 
Further  development  of  automated  decision  support  for  tactical  decisionmaking 
is  warranted. 


Cooper,  G.  E.,  Moore,  M.  H.t  and  Halpin,  S.  M.  MOVANAID :  An  interactive 
aid  for  analysis  of  movement  capabilities.  Technical  Paper  305, 

October  1978.  (AD  A064  276) 

A  computer-drive  interactive  aid  for  movement  analysis,  called  MOVANAID, 
has  been  developed  and  is  described  in  this  report.  The  aid,  which  has  been 
designed  to  be  of  assistance  in  the  performance  of  certain  Army  intelligence 
processing  tasks  in  a  tactical  environment,  can  compute  fastest  travel  times 
and  paths  through  road  networks  for  military  units  of  various  types,  as  well 
as  fastest  times  in  which  simultaneous  maneuvers  of  certain  types  can  be 
completed.  Military  situations  in  which  the  aid  might  be  useful  are  dis¬ 
cussed,  and  the  manner  in  which  users  interact  with  the  aid  is  described. 
Details  about  the  analytic  basis  upon  which  the  aid  has  been  constructed, 
together  with  a  discussion  of  the  method  in  which  the  aid  has  been  computer- 
implemented,  are  also  given.  Ideas  for  ways  in  which  the  capabilities  of 
the  aid  can  be  extended  are  discussed. 


Kibler,  A.  W. ,  Watson,  S.  R. ,  Kelly,  C.  W. ,  III,  and  Phelps,  R.  H.  A  pro¬ 
totype  aid  for  evaluating  alternative  courses  of  action  for  tactical 
engagement.  Technical  Report  TR-78-A38,  November  1978.  (AD  A064  275) 

This  technical  report  describes  a  decision-aiding  technique  developed 
to  assist  division-level  commanders  and  their  staffs  in  choosing  among  al¬ 
ternative  courses  of  action  for  tactical  engagement.  By  employing  princi¬ 
ples  of  multi-attribute  utility  assessment  methodology,  a  two-level  model 
consisting  of  five  general  categories  (terrain,  own  forces,  enemy  forces, 
weather,  and  risk)  and  24  factors  was  developed  and  implemented  on  an  IBM 
5100  computer.  In  applying  the  decision  aid,  the  user  is  required  to  score 
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each  of  the  alternative  courses  of  action  on  each  factor  and  to  assign  a 
weight  indicating  the  importance  of  each  factor  in  discriminating  among  the 
alternatives.  A  simple  algorithm  is  used  to  calculate  a  weighted  score  for 
each  course  of  action,  the  highest  score  being  an  indication  of  the  pre¬ 
ferred  course  of  action.  A  sensitivity  analysis  provides  a  measure  of  the 
robustness  of  the  scores  and  weights  assigned  by  the  user. 

The  aid  provides  a  means  of  encouraging  detailed  consideration  of  all 
factors  bearing  on  a  tactical  choice,  a  method  for  integrating  subjective 
assessments  of  decision  factors,  and  a  means  of  explicitly  communicating 
results  of  the  decision  process. 


Parrish,  R.  N.,  and  Stevens,  G.  W.  Development  of  data  processing  strate¬ 
gies  for  potential  applications  in  the  Tactical  Operations  System  and 
other  tactical  data  systems.  Research  Note  80-33,  February  1979. 

This  report  describes  research  performed  by  the  System  Development  Cor¬ 
poration  (5DC)  to  develop  improved  procedures/data  processing  methodologies 
for  forecasting  battlefield  personnel  attrition  and  for  reporting  logistics 
status.  Two  active  Army  divisions  provided  information  vis-a-vis  how  the 
two  lattermentioned  functions  are  now  performed  manually.  This  information 
provided  the  basis  for  the  development  of  a  data  processing  strategy  concept 
which,  if  implemented  in  an  automated  tactical  data  system,  should  greatly 
improve  the  accuracy  and  timeliness  with  which  the  functions  addressed  can 
be  performed.  An  experiment  conducted  with  Command  and  General  Staff  Col¬ 
lege  students  confirmed  that  the  timeliness  and  accuracy  of  the  target  func¬ 
tions  would  be  significantly  improved. 


Hoblltzell,  C.  M. ,  and  Phelps,  R.  H.  Computerized  procedures  guide:  De¬ 
cision  aid  for  evaluating  Enemy  Courses  of  Action  (ENCOA) .  Working 
Paper  HF  80-7,  January  1980. 

This  paper  provides  guidelines  for  use  of  a  computer  implementation  of 
the  aid  described  in  Kibler,  A.  W.,  TR-78-A38,  Decision  Support. 


Hoblitzell,  C.  M. ,  and  Phelps,  R.  H.  Manual  procedures  guide:  Decision 
aid  for  evaluating  Enemy  Courses  of  Action  (ENCOA) .  Working  Paper 
HF  80-8,  January  1980. 

This  paper  provides  guidelines  for  use  of  a  manual  (i.e.,  noncomputer¬ 
ized)  version  of  the  aid  described  in  Kibler,  A.  W.,  TR-78-A38,  Decision 
Support . 


Parrish,  R.  N.,  Stevens,  G.  W. ,  and  Stewart,  S.  R.  A  resource  planning  aid 
for  assessing  the  personnel  and  logistics  implications  of  tactical 
operations.  Research  Report  1295,  November  1980. 

This  report  describes  the  research  performed  by  the  System  Development 
Corporation  (SDC)  to  develop  data  processing  techniques  to  enhance  the  per¬ 
formance  of  personnel  administration  and  logistics  functions  at  corps  and 
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subordinate  echelons.  The  research  focused  on  administrative/logistics  func¬ 
tions  that  relate  directly  to  tactical  command  and  control.  European  and 
CONUS  Corps  provided  principal  sources  of  data  used  to  isolate  Resource 
Planning  as  the  function  most  needing  support.  A  conceptual  job  aid  was 
then  developed  to  support  the  area  under  investigation  and  tested  using  a 
modified  CGSC  scenario  to  determine  feasibility.  An  evaluation  briefing 
on  the  methodology  was  prepared,  in  concert  with  CACDA  and  ARI,  and  was  pre¬ 
sented  to  staff  officers  in  the  European  Theater  to  determine  its  ability 
to  fulfill  the  needs  of  the  user.  The  major  finding  of  the  evaluation  is 
that  the  methodology  is  needed  and  should  be  considered  for  implementation 
in  European  Theater. 


Phelps,  R.  H. ,  Halpin,  S.  M. ,  and  Johnson,  E.  M.  A  decision  support  framework 

for  decision  aid  designs.  Technical  Report  504,  in  press  (January  1981). 

A  Decision  Support  Framework  is  presented  (Figure  1)  which  serves  two 
purposes:  first,  to  organise  and  integrate  various  decision  aids  according 
to  their  function,  and  secondly  to  provide  the  decision  aid  designer  with  a 
systematic  context  in  which  to  develop  decision  aids  as  well  as  examine  which 
aspects  of  the  decision  problem  would  most  benefit  from  decision  aiding.  The 
main  components  of  the  framework  are  discussed  in  detail  with  Army  intelli¬ 
gence  decision-making  examples:  (1)  analysis  of  the  decision  requirements; 

(2)  development  of  decision  aids  to  provide  the  decisionmaker  with  informa¬ 
tion  as  well  as  tools  for  evaluating,  weighting,  and  integrating  the  infor¬ 
mation  to  make  a  decision;  and  (3)  evaluation  of  the  success  of  the  decision 
aids  in  leading  to  a  logical,  rational  decision. 
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Bowen,  R.  J. ,  Feehrer,  C.  E.,  Nickerson,  R.  S.,  and  Triggs,  T.  J.  Computer- 
based  displays  as  aids  in  the  production  of  Army  tactical  intelligence. 
Technical  Paper  258,  February  1975.  (AD  A007  819) 

Man-machine  interactive  computer-based  display  techniques  were  evalu¬ 
ated  for  their  automatic  data  processing  (ADp)  applications  in  combat  intel¬ 
ligence.  Analysis  of  a  tactical  exercise  helped  conceptualize  and  formulate 
display  formats  and  procedures  for  tactical  intelligence  production.  Methods 
are  suggested  for  transposing  military  exercises  into  data  base  structures 
which  support  computer-generated  situation  displays.  Concepts  of  mass  and 
movement  analyses  using  interactive  displays  are  developed.  ADP  problems 
of  software,  memory,  and  information  organization  are  considered.  Implemen¬ 
tation  of  a  prototype  system  will  validate  the  analytic  methods  developed 
here  and  explore  other  techniques  which  capitalize  on  the  combination  of  the 
computer's  ability  to  generate  complex  displays  and  the  analyst's  control  of 
specific  displays  to  be  generated. 


Moses,  F.  L.,  and  Vande  Hei,  R.  P.  A  computer  graphic-based  aid  for  analyz¬ 
ing  tactical  sightings  of  emmy  forces.  Technical  Paper  287,  January 
1978.  (AD  A049  578) 

Computer  and  computer-driven  graphics  are  being  explored  for  potential 
utilization  by  the  battlefield  staff.  One  such  effort  is  designed  to  assist 
in  detecting  patterns  by  replacing  manual  plots  of  sightings  with  flexible 
computer  graphic  techniques  and  associated  algorithms.  An  interaction  of 
the  analyst  with  these  displays  and  algorithms  for  analyzing  sightings  forms 
the  basis  for  the  present  evaluation  of  a  battlefield  interpretation  aid. 

Questions  were  asked  about  direction,  speed  of  movement,  and  changes  in 
location  of  battlefield  activity  in  a  classical  division  attack  scenario. 

Six  "aided"  participants  derived  answers  by  specifying  activity  on  a  graphic 
display  from  which  the  computer  calculated  distances,  speeds,  etc.  Six 
"unaided"  participants  derived  answers  from  the  displays  without  the  use  of 
computerized  calculations.  Responses  in  the  aided  condition  were  substan¬ 
tially  more  accurate  than  responses  in  the  unaided  condition.  Measures  of 
time  did  not  meaningfully  discriminate  between  the  conditions.  Accuracy 
results  suggest  that  analyst-controlled  computerized  algorithms  should  be 
used  for  determining  enemy  patterns  for  computer  graphic  displays  of  sight¬ 
ings.  The  aid  could  be  improved  by  better  adaptation  to  equipment,  thorough 
training,  and  design  refinements.  However,  results  are  sufficiently  promis¬ 
ing  to  suggest  provision  for  accommodating  mass/movement  and  related  temporal/ 
spatial  analysis  algorithms  in  requirements  documents. 


Irving,  G.  W.,  Farrell,  L.  M.,  and  Lindquist,  G.  Modeling  of  tactical  events 
by  interactive  graphics:  Approach,  interface  design,  and  system  design. 
Technical  Report  TR-78-A40,  February  1978.  (AD  A064  324) 

The  potential  role  for  computer  graphics  in  interactive  tactical  model¬ 
ing  was  investigated  using  a  four-color  vector  graphics  display.  The  display 
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enabled  a  user  to  define  terrain  features,  order  of  battle,  and  the  move¬ 
ments  and  actions  of  units.  Interactive  devices  included  a  light  pen,  track 
ball,  and  function  keyboard.  The  model  emphasised  tactical  planning,  al¬ 
though  similar  techniques  could  be  used  for  tactical  analysis.  A  simplified 
intelligence  planning  task  was  developed  to  demonstrate  the  concept  of 
graphics  modeliM.  In  it,  an  intelligence  performance  model  simulated  the 
detection  and  classification  performance  of  patrol  units.  A  movement/ter¬ 
rain  model  guided  user-postulated  movements  of  units,  and  probability  al¬ 
gorithms  determined  performance  of  units.  The  research  showed  that  graphic 
simulations  defined  by  the  user  can  be  developed  to  help  conceive  and  evalu¬ 
ate  potential  combat  activity. 


Irving,  G.  W.,  Farrell,  L.  M. ,  and  Lindquist,  G.  Modeling  of  tactical  events 
by  interactive  graphics,  Volume  2t  Software  documentation  and  algo¬ 
rithms  (Appendixes),  Technical  Report  TR-78-A40,  December  1978. 

Volume  I,  TR-78-A40,  described  a  procedure  for  defining  Army  tactical 
models  on  a  four-color  vector  graphics  display  for  intelligence  collection 
planning  and  intelligence  data  analysis.  The  models  defined  by  a  user  at 
the  display  consist  of  representations  of  terrain,  order  of  battle  for  oppos¬ 
ing  forces,  movement  and  actions  of  opposing  forces,  and  movement  of  friendly 
intelligence  collection  units. 

Volume  II  contains  the  algorithms  used  in  the  models  and  computer  pro¬ 
gram  listings  of  all  software  developed. 


Schechterman,  M.  D.,  and  Levi,  L.  R.  Dynamic  displays  for  tactical  planning. 

Volume  Ills  Software  documentation.  Research  Note  80-9,  December  1979. 

Volume  III  of  the  three-volume  set  contains  detailed  documentation  of 
computer  software  used  to  support  research  on  two-sided,  user-controlied, 
dynamic  wargaming.  The  current  volume  is  intended  for  systems  programmers/ 
technical  personnel  who  are  interested  in  specifics  of  implementation.  Com¬ 
plete  program  listings  are  provided.  Volume  7 ,  ARI  Research  Report  1247, 
provides  a  functional  description  of  the  research  for  Army  users  of  the  con¬ 
cepts.  Volume  II,  ARI  Technical  Report  455,  is  intended  for  specialists  in 
research  and  development  of  interactive  graphics  for  battlefield  applications. 


Walsh,  D,  H.,  Rebane,  G.  J.,  Levi,  L.  R. ,  and  Schechterman,  M.  D.  Dynamic 
displays  for  tactical  planning.  Volume  2:  Technical  description. 
Technical  Report  455,  December  1979. 

Volume  II  of  the  three-volume  set  is  a  technical  description  of  an  ex¬ 
ploratory  application  of  computer  graphics  with  animation  capabilities  for 
two-sided,  user-controlled,  dynamic  wargaming.  This  description  is  intended 
for  readers  with  specialized  interests  in  research  and  development  of  inter¬ 
active  graphics  for  battlefield  applications.  The  project  emphasizes  that 
a  battlefield  planner/analyst  can  work  in  harmony  with  computer  graphics  to 
structure  and  analyze  battlefield  situations. 
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Preliminary  procedures  were  developed  that  allow  noncomputer  personnel 
to  reate  battlefield  situations  on  displays  and  to  assess  changing  events 
us  g  dynamic  replays  and  computer  calculations  of  possible  outcomes.  Spe¬ 
cial  displays  allow  the  planner  to  see  how  terrain  affects  unit  mobility  and 
combat  effectiveness.  In  addition,  the  use  of  dynamic  replays  of  events  and 
likely  engagements  helps  the  planner  to  interpret  time  and  space  relation¬ 
ships  within  scenarios.  The  aid  has  potential  for  allowing  rapid  evaluation 
of  "what  if?”  battlefield  questions. 


Rebane,  G.  J.,  Walsh,  D.  H.,  Moses,  F.  L.,  Schechterman,  M.  D.,  and  Levi, 

L.  R.  Dynamic  displays  for  tactical  planning.  Volume  Is  User  oriented 
description.  Research  Report  1247,  April  1980. 

Volume  I  of  a  three-volume  set  describes  an  exploratory  application  of 
computer  graphics  with  animation  capability  for  two-sided,  user-controlled, 
dynamic  wargaming.  The  description  is  intended  for  Army  managers,  command 
staffs,  and  other  potential  users  of  the  concepts.  The  project  emphasizes 
that  a  battlefield  planner/analyst  can  work  in  harmony  with  computer  graphics 
to  structure  and  analyze  battlefield  situations. 

Preliminary  procedures  were  developed  that  allow  noncomputer  personnel 
to  create  battlefield  situations  on  displays  and  to  assess  changing  events 
using  dynamic  replays  and  computer  calculations  of  possible  outcomes.  Spe¬ 
cial  displays  allow  the  planner  to  see  how  terrain  affects  unit  mobility  and 
combat  effectiveness.  In  addition,  the  use  of  dynamic  replays  of  events 
and  likely  engagements  helps  the  planner  to  interpret  time  and  space  rela¬ 
tionships  within  scenarios.  The  aid  has  potential  for  allowing  rapid  evalu¬ 
ation  of  "what  if?"  battlefield  questions.  Volume  II,  ARI  Technical  Report 
455,  is  intended  for  readers  with  specialized  interests  in  research  and  de¬ 
velopment  of  interactive  graphics  for  battlefield  applications.  Volume  III, 
ARI  Research  Note  80-9,  contains  detailed  documentation  for  systems  program¬ 
mers/technical  personnel  who  are  interested  in  specifics  of  implementation. 
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McKendry,  J.  M. ,  Wilson,  R.  C.,  Maca,  D.  J. ,  and  Bakar,  J.  D.  Application 

of  a  method  for  datarmining  information  requirements  in  a  fiald  Army. 

Technical  Papar  247,  August  1973.  (AD  767  262) 

Tha  Army  is  devaloping  a  number  of  automatic  information  systems  to 
axpadita  tactical  staff  operations  in  tha  fiald,  by  providing  vital  infor¬ 
mation  to  staff  officers  in  sufficient  detail  at  appropriate  times.  Tha 
research  presented  here  provided  techniques  and  data  for  more  effectively 
evaluating  one  of  these  systems,  the  Tactical  Operations  System  (TQS)  in 
tha  Seventh  Army  in  Europe,  and  its  user  information  requirements. 

General  interest  patterns  were  found  to  be  almost  exclusively  a  func¬ 
tion  of  the  staff  element  in  which  a  person  served.  G2  and  G3  personnel 
required  the  greatest  variety  of  information,  CBRE  the  least.  There  was 
general  agreement  on  which  items  were  most  important  within  each  staff  ele¬ 
ment,  and  general  disagreement  between  staff  elements,  which  indicates  that 
the  specific  interest  patterns  for  each  element  should  be  vi  ved  separately 
in  determining  staff  information  requirements. 

The  methodology  and  techniques  employed  here  for  determining  user  in¬ 
formation  requirements  are  capable  of  producing  criteria  for  measuring  how 
thoroughly  information  is  disseminated  in  a  given  system.  Beyond  this  im¬ 
mediate  use,  such  methodology  and  techniques  could  prove  useful  for  gener¬ 
ating  staff/user  information  requirements  for  a  variety  of  Army  tactical  data 
systems,  which  in  turn  could  influence  data  base  structure  and  display  design. 


S'crub,  M.  H.,  and  HcConnaughey ,  P.  Tactical  planning  (offensive  and  defen¬ 
sive)  minimum  essential  information  requirements.  Research  Memorandum 
74-7,  October  1974.  (AD  A076  726) 

This  research  identified  minimum  essential  information  requirements 
for  offensive  and  defensive  tactical  planning.  The  basis  of  identification 
was  the  nunbor  of  officers  requesting  the  information  during  the  course  of 
offensive  or  defensive  scenario  play.  While  the  controlled  environment  of 
the  experiment  placed  some  restrictions  on  the  genaraiizability  of  the  re¬ 
quirements  identified,  the  results  reflect  a  bodv  of  information  items  con¬ 
sistently  requested  by  experienced  Army  officers.  The  hierarchical  structure 
of  the  data  base  permitted  the  delineation  of  these  requirements  by  category 
and  leveA  of  detail.  The  information  should  serve  as  a  basis  of  comparison 
and  synthesis  with  information  requirements  identified  by  other  means,  such 
as  surveys  and  ad  hoc  analytic  groups. 


Mace,  D.  >7 , ,  and  Baker,  J.  D.  Information  requirements  in  a  field  Army. 

Working  Paper  HF-80-1,  November  1979. 

As  automated  Command  and  Control  systems  became  more  prevalent  in  the 
early  1970’s  the  need  fur  an  accurate  description  of  tactical  information 
requirements  increased.  A  survey  questionnaire  containing  61  of  the  infor¬ 
mation  items  most  frequently  disseminated  in  7th  Army  exercises  was  ad¬ 
ministered  to  86  senior  staff  officers  of  7th  Army's  Corps  and  Divisions. 
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The  respondents  wart  asked  to  indicata  the  items  moat  important  to  the: . 
elements'  mission  and  to  indicata  the  level  of  information  detail  re¬ 
quired.  Results  identified  items  of  specific  interest  to  individual 
staff  elements#  as  well  as  items  and  groups  of  items  of  ccmmon  interest 
to  two  or  more  staff  elements.  The  level  of  detail  required  was  essentially 
the  same  for  all  echelons  and  higher  than  the  field  manual  racornnendations 
for  all  echelons. 


Strub ,  N.  H.  Tactical  planning  (offensive  and  defensive)  information  re¬ 
quirements  i  Gjmparison  of  survey  questionnaire  and  laboratory  exer¬ 
cise  data.  Working  Paper  HF-80-2.  November  1979. 

This  research  attempted  to  assess  the  validity  of  defining  command 
staff  information  requirements  by  means  of  a  survey  questionnaire.  Predic¬ 
tions  concerning  the  type,  amount,  and  detail  of  information  required  in 
two  division- level  tactical  scenarios,  obtained  by  questionnaire,  were 
compared  with  information  requested  by  officers  actually  playing  these  two 
scenarios  in  the  role  of  G3.  Results  showed  that  predictions  based  upon 
opinions  of  experienced  personnel  in  a  desk  setting  (such  as  those  obtained 
by  the  questionnaire)  did  well  for  the  types  of  information  required,  although 
there  was  a  tendency  to  assume  that  much  qreater  levels  of  information  detail 
were  required  than  were  actually  used.  This  finding  is  consistent  with  pre¬ 
vious  research.  The  results  indicate  that  if  a  determination  of  minimum 
information  requirements  is  critical  the  requirements  are  better  obtained 
through  the  use  of  Tactical  Exercise  Simulators  than  by  introspective 
techniques. 
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Saalberg,  J.  J.,  Miller,  J.  R. ,  Frieac,  T.  L. ,  and  Keegan,  C.  A.  Explora¬ 
tory  examination  of  purge  techniques.  Technical  Report  77-A21.  Novem¬ 
ber  1977.  (AD  A047  354) 

The  problem  of  overload  in  tactical  information  systems  can  be  reduced 
by  purging— freeing  a  tactical  data  base  of  useless,  redundant,  outdated, 
and  incorrect  information.  This  report  analyses  the  role  of  information 
in  decisionmaking  and  examines  techniques  for  identifying  a  decisionmaker's 
information  needs.  Current  purging  procedures  and  division  level  informa¬ 
tional  needs  for  land  combat  are  reviewed  and  potential  criteria  developed 
for  identifying  information  essential  for  task  performance  in  the  Division 
Tactical  Operations  Center.  Rules,  techniques,  and  operative  procedures  are 
suggested  which  can  be  employed  to  manage  and  control  TOS  data. 


Human  Factors  Technical  Area,  Fort  Leavenworth  Field  Unit,  Tos  Manager's 
Office,  and  Vector  Research,  Tnc.  Guidelines  for  information  manage¬ 
ment  for  the  Tactical  Operations  System  (TOS) t  Provisional  standing 
Operating  Procedures  (SOP).  Working  Paper  HF  79-01,  May  1979. 

Preliminary  results  of  the  project  described  in  Research  Report  1228 
were  used  to  establish  a  provisional  SOP  for  information  management  within 
tos.  This  SOP  is  outlined  in  this  paper. 


Vector  Research,  Inc.  A  design/decision  aid  for  the  Tactical  Operations 
System  (TOS) .  Working  Paper  HF  79-02,  May  1979. 

As  described  in  Research  Report  1228,  an  aid  was  developed  to  support 
analyses  of  information  flow  in  TOS  and  similar  systems.  This  paper  describes 
the  mathematical  basis  of  the  aid. 


Geiselman,  R.  E. ,  and  Samet,  M,  G.  Information  summarisation  in  a  corps- 

level  scenario.  Technical  Report  385,  October  1979.  (AD  A078  021) 

This  research  obtained  data  to  develop  guidelines  for  summarising  TOS 
message  content,  particularly  tactical  intelligence  data.  Sixteen  staff 
offic<  .*s  were  asked  to  read  a  description  of  a  tactical  scenario  and  to  ex¬ 
amine  30  enemy  situation  data  messages  describing  the  beginning  of  an  enemy 
border  crossing  and  attack.  Each  officer's  task  was  to  summarize  the  tacti¬ 
cal  information  contained  in  the  messages  in  preparation  for  a  3-minute 
briefing  of  the  G2.  The  16  handwritten  summaries  were  typed  and  evaluated 
by  five  military  raters. 


Vector  Research,  Inc.,  and  Perceptronics,  Inc.  Information  management  for 
the  '•'actical  Operations  System  (TOS)  .  Research  Report  1228,  October 
1979.  (AD  A081  9 ’4) 

This  executive  summary  presents  synopses  of  three  items  resulting  from 
the  first  phase  of  a  project  to  establish  guidelines  for  a  Standing  Operating 
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Procedure  (SOP)  for  managing  information  at  the  division  level  within  the 
Tactical  Operations  System  (TOS) ,  an  automated  staff-operations  information 
system.  Two  research  products  were  developed,  a  provisional  SOP  and  a  TOS 
design/decision  aid  model.  ARX  Technical  Report  385,  the  third  item,  pre¬ 
sents  the  qualities  found  to  be  characteristic  of  a  "good"  summary  of  bat¬ 
tlefield  intelligence  information. 


Blum,  R.  w.,  Callahan,  C.,  Cherry,  W.  P.,  Kleist,  D.,  Touma,  0.,  and  witus,  G. 

Information  management  for  an  automated  battlefield  command  and  con¬ 
trol  systems  Executive  summary.  Research  Report  1249.  May  1980. 

This  executive  summary  presents  synopses  of  seven  documents  produced 
in  the  second  phase  of  a  project  to  develop  information  management  concepts 
and  procedures  for  automated  battlefield  command  and  control  (ABCC)  systems. 
ARI  Research  Report  1248  describes  considerations  in  and  procedures  for  the 
management  of  contemporary  ABCC  systems.  ARI  Technical  Report  458  presents 
an  analysis  of  procedures  for  the  extraction,  summarisation,  and  presenta¬ 
tion  of  critical  information. 

ARI  Research  Note  80-12  describes  an  analysis  of  information  flow  in 
the  Tactical  Operations  System  (TOS) ,  an  example  ABCC  system.  ARI  Research 
Note  80-13  describes  the  mathematical  model  used  in  the  information  flow 
analysis.  ARI  Research  Notes  80-14  and  80-15  describe  the  operation  pro¬ 
gramming  of  the  computer  programs  implementing  the  model.  ARI  Working  Paper 
HF-80-XX  discusses  design  issues  associated  with  the  emerging  All-Source 
Analysis  System  (ASAS)  concept. 


Callahan,  C.,  Blum,  R.  w. ,  Witus,  G.,  end  Graulich,  M.  Guidelines  for 

managing  the  flow  of  information  in  an  automated  battlefield  command 
and  control  system.  Research  Report  1248,  May  1980. 

This  report  documents  research  directed  toward  the  development  and 
evaluation  of  guidelines  for  Standing  Operating  Procedures  (SOP)  for  manag¬ 
ing  information  flow  in  automated  battlefield  command  and  control  systems. 
This  research  was  performed  during  the  second  phase  of  a  project  to  develop 
information  management  concepts  and  procedures  for  tactical  operations  sys¬ 
tems.  Three  stages  of  SOP  guideline  development  are  discussed:  (1)  identi¬ 
fying  critical  problems)  (2)  developing  candidate  management  procedures  to 
address  the  critical  problems)  and  (3)  eva.'.uat-i  ng  the  efficacy  and  side  ef¬ 
fects  of  the  candidate  procedures. 


Cherry,  W.  P.  All  source  analysis  system:  Design  issues.  Research  Report 

1250,  May  1980. 

Experience  with  the  development  of  battlefield  automated  systems,  e.g., 
TSQ-73,  TACFIRE,  CS3,  and  TOS,  has  not  been  uniformly  satisfactory.  Schedules 
have  slipped,  costs  have  increased,  and  performance  standards  have  been  dif¬ 
ficult  to  achieve.  A  significant  reason  for  this  is  that  these  command  and 
control  systems  operate  in  a  process  which  is  highly  variable,  dependent 
upon  individuals,  and  not  completely  understood.  However,  problems  have 
also  arisen  with  the  interfaces  among  these  systems  and  between  the  systems 
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and  other  components  of  the  total  command  and  control  process.  Paramount 
among  these  problems  is  the  fact  that  existing  communications  will  not  sup¬ 
port  input,  output,  and  exchange  of  data.  This  particular  problem  As  fur¬ 
ther  complicated  by  the  dynamics  of  radio-electronic  combat  on  the  modern 
battlefield  roquiring  high  levels  of  resistance  to  j aiming  and!  intercept. 

'  The  AS AS  system  currently  envisioned  provides  capacity  but  not  cover¬ 
age  nor  coranunications  sufficiently  adequate  to  fulfill  its  mission.  It  is 
possible  that  coverage  and  communication  problems  will  be  solved  before  the 
ASAS  is  fielded.  The  primary  problems  in  the  ASAS/ECS2  interface  stem  from 
a  lack  of  understanding  of  commanders'  information  needs  and  their  relative 
values,  Beyond  this,  problems  with  communications  and  interfaces  among  the 
components  of  the  ECS2  system  will  continue  to  pose  a  major  stumbling  block 
to  successful  systems  development,  both  for  the  ASA,  and  the  ECS2  as  a  whole. 


Samet,  M.  G. ,  and  Geiselman,  R.  E.  Guideline  development  for  summarisation 

of  tactical  data.  Technical  Report  458.  May  1980. 

The  present  study  was  conducted  to  validate  and  expand  the  general iza- 
bility  of  a  set  of  guidelines  for  summarising  tactical  message  content,  and 
to  develop  guidelines  for  conducting  a  manual  purge  of  a  message  file. 
Thirty-two  Army  staff  officers  studied  a  scenario  and  30  enemy-situation- 
data  (ESD)  messages  describing  an  enemy  defensive  operation.  Their  task 
was  to  summarise  the  tactical  information  in  the  form  of  an  intelligence 
briefing.  Half  the  officers  were  supplied  with  three  general  guidelines  to 
follow  in  preparing  the  summary;  "(1)  provide  an  interpretation  of  the 
data,  (2)  give  a  dynamic  portrayal,  and  (3)  use  conversational  style.'1 
After  the  summarisation  task,  the  participants  were  asked  to  delete  15  of 
the  30  messages  and  to  rank-order  those  retained  on  the  basis  of  importance. 
The  32  summaries  were  evaluated  by  seven  military  raters  in  terms  of  quality 
of  content,  interpretation,  accuracy,  organization,  and  style. 

The  raters  judged  the  summaries  prepared  with  the  aid  of  the  guidelines 
to  be  significantly  "better"  summaries  on  an  overall  evaluation  scale  and 
on  four  of  five  qualitative  scales,  and  potent  positive  effects  were  revealed 
for  all  three  guidelines.  The  10  summaries  that  received  the  highest  overall 
evaluations  were  used  in  a  procedure  to  derive  a  general  suggested  outline 
for  describing  the  message  content  (Geiselman  s  Samet,  1979) .  The  outline 
.  ructures  information  according  to  levels  of  importance,  ranging  from  a 
description  of  immediate  enemy  threat  and  intentions  to  details  about  support 
units.  The  purging-task  data  also  revealed  general  levels  or  clusters  of 
messages  upon  which  guidelines  could  be  based  for  reducing  the  size  of  ESD 
files  by  different  amounts. 


Witus,  G.,  Blum,  R.  W.,  and  Meerschaert,  M.  Analysis  of  information  flow  in 
the  Tactical  Operations  System  (TOS) .  Research  Note  80-12,  May  1980, 

Witus,  G. ,  Blum,  R.  W. ,  and  Graulich,  M.  Description  of  the  Tactical  Opera¬ 
tions  System  information  flow  model.  Research  Note  80-13,  May  1980. 

Witus,  G. ,  Meerschaert,  M.,  and  Kleist,  D.  User's  manual  for  the  Tactical 
Operations  System  analysis  package.  Research  Note  80-14,  May  1980. 
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Witus,  G.,  Meerschaert,  M. ,  and  Kleist,  D.  Programmer's  manual  for  the 

Tactical  Operations  System  analysis  package*  Research  Note  80-15, 

May  1980. 

The  four  reports  listed  here  describe  various  aspects  of  an  analysis 
of  the  Tactical  Operations  System  (TOS) .  The  research  effort  evolved  from 
previous  work  to  develop  a  design/decision  aid  (DDA)  for  the  evaluation  of 
alternative  information  management  policies.  The  original  DDA  model  was 
concerned  exclusively  with  the  Division  Computing  Center.  The  effort  ex¬ 
panded  to  encompass  both  the  distributed  processors--the  Tactical  Computer 
Systems  and  Tactical  Computer  Terminals — and  the  supporting  communications. 
The  resulting  model  not  only  provides  a  tool  for  the  analysis  of  TOS  and 
its  component  parts,  but  also  has  the  potential  for  application  to  other 
distributed  command  support  systems  with  a  central  node  and  data  base. 

ARI  Research  Note  80-12  describes  the  results  of  an  analysis  of  the 
Tactical  Operations  System  (TOS)  communications  subsystem  performed  in  the 
second  phase  of  a  project  to  develop  information  management  concepts  and 
procedures  for  automated  battlefield  command  and  control  systems. 

Research  Note  80-13  describes  a  mathematical  model  of  the  Division 
Tactical  Operations  System  (DIV  TOS) . 

Research  Note  80-14  provides  instructions  for  the  use  and  operation  of 
a  computer  program  package  for  the  analysis  of  the  Tactical  Operations 
System  (TOS) . 

Research  Note  80-15  discusses  the  programming  details  of  the  computer 
program  package  for  the  analysis  of  the  Tactical  Operations  System  (TOS) . 
The  program  computer  code  is  included. 
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Bowen,  R.  J.  Survey  of  military  opinion  on  tactical  Order  of  Battle: 

Supporting  data  and  commentary.  Unpublished  Consulting  Research 

Memorandum ,  January  1974. 

A  four-phase  questionnaire  and  interview  program  was  conducted  to  sup¬ 
port  a  review  of  the  state-of-the-art  of  Order  of  Battle  (OB)  published  as 
Technical  Paper  265.  The  four  phases  consisted  of:  structured  interviews 
with  14  OB  specialists  at  the  U.S.  Army  Intelligence  Center  and  School;  a 
questionnaire  emphasizing  Combat  Effectiveness  completed  by  12  former  G2/S2's 
and  intelligence  operations  officers;  a  questionnaire  focused  on  Combat  Ef¬ 
fectiveness  completed  by  23  former  G3/S3's  and  field  commanders;  and  infor¬ 
mal  discussions  with  former  senior  field  commanders  on  the  subject  of  Combat 
Effectiveness.  Detailed  summaries  of  each  phase  of  this  program  are  pre¬ 
sented  with  supporting  commentary. 


Bowen,  R.  J. ,  Feehrer,  C.  E.,  Nickerson,  R.  S.,  and  Triggs,  T.  J.  Computer- 
based  displays  as  aids  in  the  production  of  Army  tactical  intelligence. 
Technical  Paper  258,  February  1975.  (AD  A007  819) 

Man-machine  interactive  computer-based  display  techniques  were  evalu¬ 
ated  for  their  automatic  data  processing  (ADP)  applications  in  combat  in¬ 
telligence.  Analysis  of  a  tactical  exercise  helped  conceptualize  and  formu¬ 
late  display  formats  and  procedures  for  tactical  intelligence  production. 
Methods  are  suggested  for  transposing  military  exercises  into  data  base 
structures  which  support  computer-generated  situation  displays.  Concepts 
of  mass  and  movement  analyses  using  interactive  displays  are  developed. 

ADP  problems  of  software,  memory,  and  information  organization  are  con¬ 
sidered  for  a  prototype  system  to  validate  the  analytic  methods  developed 
here. 


Bowen,  R.  J.,  Halpin,  J.  A.,  Russell,  P.  T.,  and  jtaniforth,  B.  J.  Tactical 
Order  of  Battle:  A  state-of-the-art  survey.  Technical  Paper  265, 
October  1975.  (AD  A018  368) 

This  report  provides  a  state-of-the-art  survey  of  tactical  Order  of 
Battle  (OB)  intelligence,  concentrating  on  the  division  OB  section  and  based 
on  official  documentation,  authoritative  reference  material,  and  the  results 
of  an  informal  questionnaire  and  interview  program  with  experienced  combat 
intelligence  and  operations  personnel.  The  survey  includes:  historical 
development  of  OB;  role,  function,  and  operations  of  the  division  OB  section; 
examination  of  the  OB  factors;  and  the  results  of  a  brief  survey  of  military 
opinions  on  OB.  OB  analysis  requires  more  varieties  of  detailed  information 
processing  than  any  other  single  area  of  combat  intelligence.  However,  the 
procedures  and  techniques  available  are  not  standardized  or  well  developed, 
and  the  OB  section  tends  to  provide  historical  reference  information  rather 
than  current  intelligence.  Specific  findings  and  recommendations  are  pre¬ 
sented  after  each  section  of  the  survey. 
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Coates,  E.  N. ,  and  McCourt,  A.  W.  A  questionnaire-based  analysis  of  Order- 

of  Battle  elements.  Technical  Paper  271,  January  1976.  (AD  A021  956) 

On  a  questionnaire  administered  primarily  to  field  grade  officers, 

1,258  respondents  rated  the  value  to  them  of  the  eight  elements  of  tactical 
Order  of  Battle  (OB)  intelligence  and  of  the  95  components  of  these  elements. 
Respondents  evaluated  how  essential  each  element  or  component  would  be  to 
their  job  in  each  of  four  scenarios  describing  different  levels  of  war,  and 
also  rank  ordered  the  elements  and  the  components  within  each  element. 

Respondents  agreed  generally  on  the  relative  values  of  OB  elements; 
ratings  of  usefulness  followed  consistent  patterns  which  were  confirmed  by 
the  forced  rankings.  Intelligence  on  enemy  Disposition  was  consistently 
rated  the  most  valuable  in  all  four  conditions  of  war:  enemy  Strength, 
Composition,  Combat  Effectiveness,  and  Tactics  were  also  consider  d  crucial. 
The  perceived  value  of  Logistics  intelligence  increased  with  increasing  in¬ 
tensities  of  war,  while  that  of  Tactics  and  Training  was  greatest  in  low- 
intensity  conditions.  Military  Intelligence  branch  officers  valued  every 
element  except  Training  more  highly  than  Combat  Arms  officers  did  while  Com¬ 
bat  Arms  officers  consistently  valued  every  element  more  than  support  branch 
officers  did.  The  results  provide  the  basis  for  further  examination  of  in¬ 
telligence  requirements  and  priorities  in  intelligence  collection,  production, 
and  dissemination  under  various  conditions  of  war. 


Johnson,  E.  M. ,  Spooner,  R.  L.,  and  Jaarsma,  D.  Tne  perception  of  tactical 

intelligence  indications  by  intelligence  officers.  Technical  Paper 

278,  April  1977.  (AD  A040  350) 

As  part  of  an  effort  to  provide  improved  techniques  and  methods  for 
intelligence  collection  planning  and  intelligence  analysis,  an  experiment 
was  conducted  to  assess  the  effectiveness  of  traditional  tactical  intelli¬ 
gence  indications  for  the  analysis  of  conventional  military  operations . 
Forty-six  captains  in  the  Intelligence  Officers  Advanced  Course  each  assumed 
the  role  of  staff  officer  in  a  G2  section  of  an  Infantry  Division  conducting 
a  mobile  defense  in  north-central  West  Germany.  Each  officer  estimated  the 
probability  that  each  of  49  separate  indications  listed  in  Field  Manual  30-5, 
Combat  Intelligence,  would  occur,  given  the  aggressor's  known  course  of 
action.  Each  indication  was  evaluated  with  four  separate  courses  of  action — 
Attack,  Defend,  Delay,  and  Withdraw.  The  estimates  made  by  individual  offi¬ 
cers  were  highly  reliable.  However,  the  variability  in  the  estimates  made 
by  different  officers  for  the  same  indication  was  extremely  high,  ;ith  an 
average  range  of  estimates  greater  than  .7  on  a  0  to  1.0  scale.  Tie  logic 
underlying  clusters  of  related  indications  could  not  be  clearly  ic.  jntified 
for  any  of  the  four  courses  of  action.  The  findings  reveal  that  current 
indications  of  conventional  military  operations  are  either  poorly  understood 
or  intrinsically  inadequate  for  use  in  contemporary  intelligence  operations, 
or  both. 


Johnson,  E.  M.  The  perception  of  tactical  intelligence  indications:  A 
replication.  Technical  Paper  282,  September  1977.  (AD  A046  954) 

This  research  replicates  an  earlier  experiment  which  showed  that  tacti¬ 
cal  intelligence  indications,  as  currently  used,  are  of  doubtful  effectiveness 
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In  the  replication,  28  intelligence  specialists  (primarily  from  the  163d 
Military  Intelligence  Battalion)  each  assumed  the  role  of  an  intelligence 
analyst  in  a  G2  section  of  an  infantry  division  conducting  a  mobile  defense 
in  north-central  West  Germany.  Each  individual  estimated  the  probability 
that  each  of  the  49  indications  of  conventional  military  operations  listed 
in  Field  Manual  30-5,  Combat  Intelligence,  would  occur,  given  a  known  ag¬ 
gressor  course  of  action.  Each  indication  was  evaluated  with  four  separate 
enemy  courses  of  action — Attack,  Defend,  Delay,  and  Withdraw.  Variability 
in  the  estimates  made  by  different  individuals  for  the  same  indication  was 
extremely  high  (greater  than  .7  on  a  0-1.0  scale  in  both  experiments).  Only 
12  indications  were  perceived  by  this  group  and  19  by  the  previous  group  as 
effective  discriminators  of  the  course  of  action  with  which  they  are  doc- 
trinally  associated.  Estimates  made  by  officers  and  enlisted  men  in  the 
replication  did  not  differ  significantly ,  nor  did  estimates  made  by  intel¬ 
ligence  specialists  in  the  present  research  differ  significantly  from  those 
made  by  student  officers  previously.  Findings  confirmed  that  current  indi¬ 
cations  of\  conventional  military  operations  are  either  poorly  understood  or 
intrinsically  inadequate  for  use  in  contemporary  intelligence  operations, 
or  both. 


Moses,  F.  L.,  and  Vande  Hei,  R.  P.  Computer  graphic-based  aid  for  analyzing 
tactical  sightings  of  enemy  forces.  Technical  Paper  287,  January  1978. 
(AD  A049  578)  . 

See  the  listing  for  this  reference  on  p.  13  of  this  report. 


Stewart,  S.  R.  Utility  of  automation  of  Order  of  Battle  and  Target  Intelli¬ 
gence  data  for  intelligence  analysis.  Research  Report  1194,  September 
1978.  (AD  A065  458) 

To  help  determine  user  requirements  for  a  Tactical  Operations  System 
(TOS) ,  software  packages  were  evaluated  for  potential  inclusion  as  require¬ 
ments  for  a  prototype  TOS.  The  first  of  these  packages — automated  Order  of 
Battle  (OB)  and  Target  Intelligence  (TI)  files — was  evaluated  in  the  present 
study.  A  series  of  intelligence  tasks  was  accomplished  by  two  groups,  each 
consisting  of  10  subjects  qualified  as  intelligence  analysts.  Both  groups 
worked  with  a  basic  simulated  TOS.  The  experimental  group  had  access  to 
automated  OB  and  TI  files;  the  control  group  U3ed  OB  and  TI  files  in  manual 
form.  Automation  of  the  files  significantly  increased  both  the  accuracy 
and  the  completeness  with  which  intelligence  requirements  were  accomplished 
but  did  not  increase  the  speed  of  completion  of  those  requirements. 


Katter,  R.  V.,  Montgomery,  C.  A.,  and  Thompson,  J.  R.  Cognitive  processes 
in  intelligence  analysis:  A  descriptive  model  and  review  of  the  lit¬ 
erature.  Technical  Report  445,  December  1979.  (AD  A086  451) 

This  descriptive  model  of  cognitive  processes  in  intelligence  analysis 
was  developed  as  part  of  an  "Investigation  of  Methodologies  and  Techniques 
for  Intelligence  Analysis."  The  approach  to  constructing  the  model  is  based 
on  the  investigation  of  analytical  processing  in  two  types  of  single  source 
analysis  (SIGINT  and  IMINT) ,  subsequently  generalizing  to  multisource 
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analysis.  For  the  purposes  of  thin  investigation,  intelligence  analysis 
was  defined  as  a  spectrum  of  analytical  and  judgmental  activities  involved 
in  the  processing  and  production  of  intelligence,  where  particular  indi¬ 
viduals  may  devote  more  or  less  time  to  different  aspects  of  such  activities 
according  to  their  individual  roles  in  the  intelligence  cycle. 

This  report  contains  a  detailed  review  of  selected  aspects  of  current 
ps  -ohological  literature,  as  well  as  the  cognitive  model  which  is  based  on 
that  literature  and  on  investigation  of  the  1MINT  and  SIGINT  production 
models.  The  cognitive  model  and  the  literature  review  focus  on  how  informa¬ 
tion  is  processed,  stored  and  retrieved  from  memory,  evaluated,  and  inte¬ 
grated  to  form  intelligence.  The  model  has  general  applicability  to  the 
full  spectrum  of  intelligence  processing  activities. 


Katter,  R.  V.,  Montgomery,  C.  A.,  and  Thompson,  J.  R.  Human  processes  in 

intelligence  analysis:  Phase  I  overview.  Research  Report  1237,  De¬ 
cember  1979.  (AD  A086  503) 

This  report  provides  an  overview  of  the  results  of  an  "Investigation 
of  Methodologies  and  Techniques  for  Intelligence  Analysis . "  The  goal  of 
this  study  was  to  develop  a  framework  for  understanding  human  processes  in 
intelligence  analysis.  An  understanding  of  these  processes  will  be  useful 
during  the  development  or  evaluation  of  training  procedures,  doctrine,  and 
system  requirements  for  automated  support  to  analysts. 

The  approach  taken  in  this  project  was  a  parallel  development  of  in¬ 
telligence  production  models  and  a  cognitive  model  of  intelligence  analysis. 
The  Imagery  Intelligence  (IMINT)  Production  Model  (ARI  Research  Report  1210) 
developed  under  this  project  and  an  existing  Signals  Intelligence  (SIGINT) 
Production  Model  were  used  as  the  basis  for  developing  a  generic  model  of 
intelligence  production,  described  in  this  overview.  The  generic  production 
model  includes  a  description  of  the  hierarchical  flow  of  intelligence  data, 
tasking,  etc.,  as  well  as  a  listing  of  critical  variables  influencing  analy¬ 
sis.  A  detailed  review  of  the  psychological  literature  described  in  ARI 
Technical  Report  445,  combined  with  the  production  models,  led  to  develop¬ 
ment  of  a  descriptive  model  of  cognitive  processes  described  in  this  over¬ 
view.  This  overview  also  explores  some  of  the  implications  of  the  cognitive 
model  for  training  developments,  system  development,  and  organizational 
control  of  intelligence  production.  The  cognitive  model  will  be  extended 
and  these  implications  will  be  examined  in  greater  detail  during  Phase  II 
of  this  project. 
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Baker,  J.  D. ,  McKendry,  J.  M.,  and  Mace,  D.  J.  Certitude  judgments  in  an 

operational  environment.  Technical  Research  Note  200,  November  1968. 

(AD  681  232) 

This  report  describes  human  performance  research  conducted  at  the  ARI 
Field  unit  in  Europe  in  connection  with  early  development  of  the  experi¬ 
mental  Tactical  Operations  System  (TOS) .  Major  focus  was  placed  upon  the 
certitude  judgment  process  within  the  TOS.  In  the  intelligence  cycle,  G2 
spot  reports  include  subjective  estimates  of  information  expressed  in  a 
standard  rating  form.  All  messages  (N  -  2,039)  filed  by  two  divisions  of 
one  corps  during  the  7-day  field  exercise  were  examined  for  presence  of  the 
required  ratings  of  reliability  of  the  source  and  accuracy  of  the  informa¬ 
tion  being  sent.  Assessment  was  made  of  the  quantitative  and  qualitative 
improvement  that  could  be  expected  under  operating  conditions  within  a  com¬ 
puterized  TOS.  Observations  during  the  field  exercise  revealed:  (1)  G2 
spot  reports  constituted  70%  of  all  messages;  (2)  omission  of  the  required 
source  reliability  and  accuracy  ratings  occurred  in  50%  of  the  spot  reports 
(attributed,  in  part,  to  unavailability  of  standard  report  forms) »  (3)  re¬ 
liability  and  accuracy  ratings  were  noticeably  interdependent  (distributions 
of  the  values  indicated  that  senders  tended  to  assign  high  scale  values  to 
ratings) ;  (4)  analysis  of  the  rating  scales  revealed  the  source  reliability 
scale  to  be  unidimensional  and  the  accuracy  scale  to  be  multidimensional. 
Findings  point  to  substantial  increase  in  quantity  of  certitude  data  avail¬ 
able  for  decisionmaking  at  all  levels  of  command  by  introduction  of  the 
automated  TOS. 


Johnson,  E.  M.  Numerical  encoding  of  qualitative  expressions  of  uncer¬ 
tainty.  Technical  Paper  250,  December  1973.  (AD  780  814) 

Judgments  of  the  probable  accuracy  of  intelligence  data  and  products 
are  integral  to  the  intelligence  process.  Accurate  communication  of  the 
uncertainty  implicit  in  these  judgments  is  essential  to  the  effective  pro¬ 
duction  and  utilization  of  intelligence.  Twenty-eight  subjects,  14  U.S. 
Army  enlisted  men  and  14  extension  college  students,  numerically  encoded 
on  a  0  to  100  scale  each  of  15  probability  phrases  in  each  of  3  sentence 
contexts.  The  15  phrases  systematically  covered  a  wide  range  of  proba¬ 
bilistic  meaning  and  the  3  sentence  contexts  involved  a  weather  forecast, 
a  prediction  of  personal  success,  and  an  intelligence  report.  There  were 
no  significant  differences  in  the  encoding  of  probability  phrases  into 
numerical  equivalents  among  the  3  sentence  contexts ,  between  enlisted  per¬ 
sonnel  and  college  students  or  as  a  function  of  age,  sex,  or  educational 
level.  Subjects  were  relatively  consistent  in  their  own  encoding  of  given 
phrases,  but  differed,  often  radically,  from  other  subjects.  Cluster 
analysis  of  the  numbers  assigned  by  subjects  indicated  an  underlying 
asymmetric  probability  scale  comprised  of  a  small  number  of  intervals. 
Further  research  should  investigate  the  use  of  standardized  lexicons  with 
a  small  r umber  of  expressions  or  the  direct  use  of  numerical  scales. 
Numerical  scales  appear  to  be  the  most  promising  and  would  facilitate 
the  use  of  the  tools  of  probability  theory  and  decision  theory  in  intelli¬ 
gence  analysis. 
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Samet,  M.  G.  Sub-jective  interpretation  of  reliability  and  accuracy  scales 
for  evaluating  military  intelligence.  Technical  Paper  260,  January 
1975.  (AD  A003  260) 

(Also,  Quantitative  interpretation  of  two  qualitative  scales  used  to 
rate  military  intelligence.  Human  Factors,  1975,  17 ,  192-202.) 

This  report  is  part  of  a  broader  effort  to  investigate  human  perfor¬ 
mance  in  judgmental  tasks  and  to  determine  ways  of  improving  such  perfor¬ 
mance,  particularly  as  it  affects  data  input  to  tactical  intelligence  in¬ 
formation  processing.  The  research  reported  here  assessed  the  adequacy  of 
source-reliability  and  information-accuracy  rating  scales  for  conveying 
intelligence  information . 

Intelligence  officers  completed  an  original  set  and  a  replication  of 
paper  and  pencil  tasks  which  measured  their  attitudes  toward  and  knowledge 
about  the  scales;  recorded  their  judgments  as  to  which  report  in  each  of 
100  pairings  of  reports  with  different  joint  accuracy  and  reliability  rat¬ 
ings  was  more  likely  to  be  true;  asked  them  to  estimate  the  probability  that 
a  report  with  a  specific  reliability  rating  would  also  carry  a  specific  ac¬ 
curacy  rating,  and  vice  versa;  and  had  them  assign  numerical  values  repre¬ 
senting  the  probable  truth  of  reports  with  given  levels  of  reliability,  of 
accuracy,  and  reliability-accuracy  combinations.  About  one-fourth  of  the 
subjects  treated  reliability  and  accuracy  as  independent  dimensions;  the 
majority  treated  reliability  as  highly  correlated  with  accuracy,  and  their 
judgment  of  a  report's  truth  was  influenced  more  strongly  by  its  accuracy 
rating.  Numerical  (probabilistic)  interpretations  of  scale  levels  were  con¬ 
sistent  within  individuals  but  varied  widely  between  them.  Development  of 
a  new  scale  is  suggested.  The  scale  should  require  the  assignment  of  a 
quantitative  value  which  would  reflect  the  likelihood  of  a  report's  being 
true  and  be  based  on  all  available  information  including  the  reliability 
of  the  source. 


Halpin,  S.  M. ,  Moses,  F.  L,,  and  Johnson,  E.  M.  A  validation  of  the  struc¬ 
ture  of  combat  intiii.' gence  ratings.  Technical  Paper  302 ,  August  1978. 

(AD  A061  568) 

This  is  a  validation  of  a  previously  derived  four-dimensional  descrip¬ 
tive  structure  for  evaluating  the  quality  of  combat  intelligence  data. 
Fifty-six  officers  in  the  Intelligence  Officer's  Advanced  course  were  split 
into  two  groups;  one  group  rated  40  messages  from  the  Battle  of  the  Bulge 
scenario  used  in  the  original  research,  and  the  second  group  rated  40  mes¬ 
sages  from  a  scenario  based  on  a  hypothetical  mid-intensity  European  con¬ 
flict.  Each  message  was  rated  on  50  rating  scales  previously  used,  as  well 
as  on  four  new  scales  based  on  the  findings  of  the  previous  research.  Fac¬ 
tor  analyses  of  the  results  showed  that  the  structure  of  ratings  within  both 
scenarios  is  basically  the  same  as  that  derived  from  the  enlisted  analysts. 
Two  concepts.  Truth  and  Relevance/Importance/Threat,  were  major  factors  in 
the  evaluation  of  intelligence  information  in  all  cases.  Two  of  the  new 
scales,  "True"  and  "Predictable,"  reflect  the  Truth  dimension,  and  the  other 
two  new  scales,  "Important"  and  "Relevant,"  represent  the  Relevance/Impor- 
tance/Threat  dimension.  The  structure  of  intelligence  information  quality 
validated  in  the  present  study  is  not  well  represented  by  the  current  rat¬ 
ings  of  Information  Accuracy  and  Reliability  of  Source.  A  new  Accuracy 
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seals  could  capture  a  major  portion  of  the  variance  in  quality  ratings, 
and  the  other  new  scales  evaluated  here  provide  a  basis  for  further  devel¬ 
opment  of  supplementary  ratings. 


Miron,  M.  S.,  Patten,  S.  M.,  and  Halpin,  s.  M.  Determination  of  the  struc¬ 
ture  of  combat  intelligence  ratings.  Technical  Paper  286,  September 
1978.  (AD  A060  321) 

Unpublished  appendixes--A:  Test  materials.  B:  Discriminant 
analysis.  C;  Scale  factor  analyses.  D:  Report  factor  analyses. 

E:  Multiple  regression  analyses.  F:  Reports  text.  G:  KW1C. 

Hs  Frequency.  I:  Tagging.  J:  Factor  analyses  of  tagged  cate¬ 
gories  and  report  content. 

Previous  research  has  indicated  that  ratings  of  tactical  intelligence 
data  on  the  dimensions  of  accuracy  and  reliability  are  inadequate  to  convey 
the  analysts’  perception  of  the  quality  of  the  data.  Forty  messages  derived 
from  28th  Infantry  Division  files  for  the  Battle  of  the  Bulge  were  rated  by 
55  enlisted  intelligence  analysts  on  each  of  50  scales,  including  the  cur¬ 
rent  Accuracy /Re liability  scales.  These  ratings  were  factor-analyzed  to 
determine  the  underlying  judgment  structure.  Three  basic  dimensions  emerged, 
tentatively  labeled  Accuracy,  Relevance,  and  Directness.  Both  the  current 
scales  are  closely  related  to  the  Accuracy  dimension.  The  stability  of  these 
three  dimensions  remains  to  be  evaluated.  The  implications  of  this  judgment 
structure  for  quality  ratings  of  combat  intelligence  data  merit  further 
exploration . 


Phelps,  R.  H.,  Halpin,  S.  M. ,  Johnson,  E.  M. ,  and  Moses,  F.  L.  Implementa¬ 
tion  of  subjective  probability  estimates  in  Army  intelligence  procedures: 
A  critical  review  of  research  findings.  Research  Report  1242 ,  March 
1980. 

This  paper  examines  one  area  which  appears  to  degrade  the  quality  of 
tactical  intelligence:  the  formulation  and  expression  of  uncertainty  in 
both  the  intelligence  estimate  and  the  evaluation  of  information  contained 
in  spot  reports.  Two  interrelated  issues  are  examined:  (1)  current  doctri¬ 
nal  procedures  (FM  30-5)  for  both  spot  report  data  evaluation  and  the  ex¬ 
pression  of  uncertainty  in  the  intelligence  estimate  as  well  as  research  on 
the  current  use  and  problems  with  these  procedures;  and  (2)  implementation 
of  numerical  subjective  probability  estimates  to  express  uncertainty  as  an 
alternative  to  the  present  methods. 
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Using  current  procedures,  uncertainty  is  expressed  by  verbal  probability  j 

phases  (e.g.,  possible,  unlikely)  in  the  intelligence  estimate  and  by  a  7- 
point  rating  scale  based  on  verbal  phases  in  spot  report  data  evaluation. 

Available  research  clearly  shows  that  such  verbal  probability  phrases  are  ! 

extremely  ambiguous  in  interpretation.  The  use  of  numerical  subjective  un- 
certainty  in  the  intelligence  estimate  and  data  evaluation.  While  some 
questions  still  remain  concerning  the  details  of  training  and  implementation, 
our  current  state  of  knowledge  provides  a  sufficient  base  to  begin  incor¬ 
porating  subjective  probability  estimates  into  Army  doctrine  as  well  as 

practice.  5 
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Use  of  numerical  subjective  probability  estimates  is  anticipate 
decrease  the  ambiguity  in  communicating  intelligence  estimates  and  in 
evaluating  spot  reports.  Once  numerical  probabilities  have  been  incor¬ 
porated  into^intelligence  procedures,  an  additional  advantagewlllbethe^ 
ease  with  which  numerical  estimates  can  be  compared  among  personnel,  ov 
time,  cr  used  as  input9  for  automated  decision  aide. 
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Human-Computer  Interaction :  General 
Data  Entry 

Data  Entry:  System  Simulation  (MANMODEL) 

Data  Organization 

Data  Retrieval 

Graphic  Displays 

Symbology 

Topographic  Products  and  Procedures 


Human-Computer  Interaction t  General 


Human-Computer  Interaction »  General 

Grande,  T.  M.  An  application  of  human  factors  concepts  to  an  interactive 
computer iced  personnel  record-keeping  system.  Research  Report  1233, 
January  1980.  {AO  A083  160)  ~ 

This  report  describes  a  "hands-on"  demonstration  that  illustrates  how 
human  factors  principles  can  be  applied  to  the  design  of  the  human- computer 
interface  and  the  operational  procedures  of  an  interactive,  real-time  Army 
personnel  record  processing  system.  The  Standard  Installation/Division  Per¬ 
sonnel  System  (SIDPERS)  was  identified  as  a  typical  r oc or d-pro cessing  system 
suitable  for  conversion  to  an  interactive  computer  systtem.  Four  SIDPERS 
transactions  (Duty  Status,  Departure,  Arrival,  and  Grade  Change)  were  se¬ 
lected  as  representative  tasks  for  demonstrating  the  application  of  beha¬ 
vioral  principles.  The  demonstration  was  programmed  on  the  PLATO  (Programmed 
Logic  for  Automatic  Teaching  Operation)  system. 

The  demonstration  focused  on  the  interactions  between  the  computer  and 
both  an  experienced  and  an  inexperienced  procesring  clerk  of  a  unit.  The 
demonstration  also  displayed  interactions  between  a  unit;  supervisor  and  the 
computer  and  between  SIDPERS  personnel  and  the  computer.  The  demonstration 
capabilities  included  error  detection  and  correction,  definition  of  terms, 
variable  input  format,  and  efficient  information  retrieval. 

The  hands-on  demonstration  clearly  showed  the  advantages  of  an  inter¬ 
active  system  designed  for  the  user:  reduction  in  processing  time  and  num¬ 
ber  of  errors  per  transaction,  accommodation  to  users  with  differing  needs 
and  levels  of  skill,  and  increased  user  satisfaction. 


Ehrenreich,  S.  L.  Design  recommendations  for  query  languages.  Technical 

Report  484,  in  press  (November  1980) . 

The  existing  human  factors  literature  on  query  languages  is  both  sparse 
and  scattered.  This  paper  seeks  to  collect  and  review  that  literature.  The 
first  section  of  the  paper  introduces  the  subject  of  query  languages.  In 
the  second  and  third  sections,  the  topics  of  natural  and  formal  query  lan¬ 
guages  are  respectively  discussed.  These  two  types  of  query  languages  are 
reviewed  with  the  objective  of  determining  their  potential  for  expanding 
the  population  of  computer  users.  The  fourth  section  considers  some  general 
issues  pertinent  to  both  types  of  query  languages.  These  issues  include  the 
ability  of  people  to  deal  with  logical  quantifiers,  the  user's  concept  of 
data  organization,  mixed  initiative  dialogues,  and  the  use  of  abbreviations. 
Methods  for  experimentally  evaluating  specific  query  language  features  and 
research  on  person-to-person  communication  are  also  discussed  here.  To 
focus  the  findings  reported  in  the  preceding  sections,  the  fifth  section 
summarizes  the  implications  of  the  research  performed  to  date.  Next,  the 
sixth  section  presents  possible  new  research  which  would  be  of  value  to 
the  designers  of  Army  tactical  information  systems.  The  paper  concludes 
with  two  appendixes.  Appendix  A  discusses  human  factors  review  papers  con¬ 
cerned  with  the  design  of  interactive  systems.  Appendix  B  presents  a  com¬ 
pendium  of  design  recommendations  directed  toward  the  system  designer. 
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Baker,  J.  D.,  Mace,  D.  J. ,  and  McKendry,  J.  M.  The  transform  operation  in 

TOS»  Assessment  of  the  human  component.  Technical  Research  Note  312, 

August  1969.  (AD  697  716) 

Of  concern  in  this  study  were  the  human  factors  problems  involved  in 
transforming  free-English  G3  message  texts  into  rigidly  defined  format  in 
preparation  for  user  interaction  with  TOS .  In  the  experiment,  47  simple 
messages  were  given  to  14  individuals  familiar  with  TOS  or  G3  operations, 
or  both.  Half  the  men  (job-aid  group)  used  an  aid  devised  by  the  AR1  in¬ 
vestigators,  and  half  (standard  group)  used  a  "menu"  type  listing  of  avail¬ 
able  formats  in  performing  the  required  task  of  selecting  an  appropriate 
format  for  each  message.  Time  taken  to  complete  the  task  and  also  degree 
of  confidence  in  the  selection  made  were  recorded.  Findings  showed  a  mean 
time  of  approximately  50  seconds  for  message  reading  and  format  selection. 
Average  error  rate  in  format  selection  was  22%.  Mo  significant  difference 
was  obtained  in  performance  with  the  job  aid  over  t  vt  with  the  "menu"  type 
listing.  Messages  of  different  types  and  subject  m.  •  -,er  differed  in  error 
rate  of  format  selection.  Mean  expressed  certitude  of  correctness  in  for¬ 
mat  selection  was  almost  the  same  for  both  groups.  Time  and  error  rate 
data,  together  with  other  factors  such  as  screening  for  applicability  to 
TOS,  correction  of  errors,  and  translating  more  complex  tests  led  to  the 
conclusion  that  the  transformation  process  is  potentially  a  major  problem 
in  the  experimental  TOS.  Results  provided  baseline  data  for  comparative 
study  of  future  performance  and  suggested  some  approaches  to  training  and 
alternative  methods  for  the  transform  process. 


Strub,  M.  H.  Evaluation  of  man-computer  input  techniques  for  military  in¬ 
formation  systems.  Technical  Research  Note  226,  May  1971. 

(AD  730  315) 

The  present  publication  describes  speed  and  accuracy  evaluations  of 
four  configurations  of  procedures  for  inputting  information  into  a  semi- 
automated  information  processing  system.  Sixty  USMA  Prep  School  enlisted 
men  were  given  an  experimental  task  requiring  each  to  translate  35  free-text 
messages  into  computer-acceptable  terminology.  Accuracy  and  speed  of  two 
input  procedures  were  each  compared  under  two  conditions  of  verification. 

In  one  procedure,  the  subjects  translated  the  incoming  message  onto  a  paper 
format  before  transcribing  on  a  CRT  screen  (off-line) .  In  the  other  pro¬ 
cedure,  the  message  was  transcribed  directly  on  the  CRT  screen  (on-line) . 

In  the  unverified  condition,  two  men  translated  the  same  message,  compared 
their  translations,  and  resolved  differences  before  entering  the  information 
into  the  data  base.  Performance  results  under  the  four  experimental  condi¬ 
tions  were  also  compared  with  a  similar  7th  Army  TOS  procedure  in  which  a 
message  is  translated  onto  a  paper  format  and  the  unverified  message  is 
copied  on  the  CRT  screen  by  the  UIOD  (user  input-output  device)  operator. 

In  the  present  experiment,  data  input  accuracy  was  significantly  in¬ 
creased  when  free-text  messages  were  translated  directly  on  the  CRT  screen 
rather  than  iirst  filled  out  on  paper  formats  (11.2%  error  vs.  14.8%). 

Input  speed  was  practically  the  same  under  both  methods.  When  two  operators 
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chackad  aach  other's  translation  bafora  inputting  to  tha  data  base,  arror 
was  raducad  by  ona-third  (10.3%  vs.  15.7%),  but  tha  procadura  raquirad  ona- 
third  mora  time  (6.81  min.  vs.  4.98  min.).  Eithar  procadura  was  an  improve- 
mant  in  accuracy  ovar  tha  work  mathod  usad  in  tha  TOS.  Tha  prasant  study 
strongly  suggasts  that  incoming  massagas  should  ba  translatad  diractly  on 
tha* CRT  scraan.  Diract  CRT  input  would  raduca  arror  whila  eliminating  paper 
formats  and  need  for  U10D  operator  transcription.  Findings  further  suggest 
that,  when  time  and  personnel  permit,  messages  should  ba  verified  for  con¬ 
sistency  bafora  entering  the  information  into  tha  data  baaa. 


Nawrocki,  L.  H.,  Strut,  M.  H.,  and  Cecil,  R.  M.  Error  categorisation  and 
analysis  in  man-computer  communication  systems.  IEEE  Transactions 
on  Reliability.  1973,  R-22,  135-140. 

This  paper  briefly  examines  traditional  approaches  to  human  reliability 
and  presents  a  technique  which  permits  the  system  designer  to  derive  a  mutu¬ 
ally  exclusive  and  exhaustive  set  of  operator  error  categories  in  a  man- 
computer  system.  These  error  categories  are  defined  in  terms  of  process 
failures  and  provide  the  system  designer  with  a  qualitative  index  suitable 
for  determining  error  causes  and  consequences.  The  technique  is  demonstrated, 
and  the  utility  of  the  resulting  error  categories  is  evaluated  in  the  context 
of  two  studies  on  a  military  information  processing  system.  The  paper  co» - 
eludes  with  a  brief  discussion  of  detectable  and  nondetectable  errors  and  a 
suggestion  for  determining  the  impact  of  errors  on  ultimate  system  goals. 


Sidorsky,  R.  C.  Alpha-Dot:  A  new  approach  to  direct  computer  entry  of 

battlefield  data.  Technical  Paper  249,  January  1974.  (AD  774  841) 

The  purpose  of  the  present  study  was  to  determine  the  suitability  of 
the  five-key  Alpha-Dot  keyboard  for  source  data  automation  of  battlefield 
information.  The  Alpha-Dot  system  is  a  coding  technique  that  enables  people 
to  input  data  using  familiar  shapes  in  a  form  that  is  also  directly  compati¬ 
ble  with  computers  and  other  binary  data  processors.  The  technique  can  be 
used  as  the  basis  for  a  number  of  devices  and  procedures  for  the  two-way 
communication  of  information  between  man  and  machine.  Ten  enlisted  per¬ 
sonnel  entered  both  free  form  and  formatted  versions  of  simulated  enemy 
situation  spot  reports.  Learning  time  as  well  as  rate  and  accuracy  of  data 
entry  using  the  Alpha-Dot  keyboard  was  compared  to  operator  performance  us¬ 
ing  a  standard  typewriter. 

The  results  indicate  that  no  special  skill  is  required  to  operate  the 
device.  All  of  the  trainees  were  able  to  input  messages  satisfactorily 
after  2  or  3  minutes  of  instruction.  Learning  time  for  skilled  operation 
is  very  short.  Nine  of  the  10  trainees  memorized  the  character  set  within 
l*i  hours  of  practice  and  were  then  able  to  transmit  messages  without  refer¬ 
ence  to  the  guide  chart.  Rate  of  data  entry  compares  favorably  with  the 
standard  keyboard.  After  less  than  5  hours  of  practice,  free  text  messages 
were  entered  at  60%  of  each  trainee's  standard  keyboard  rate.  However, 
formatted  (TOS  type)  messages  were  transmitted  at  a  rate  equal  or  exceeding 
that  of  the  standard  keyboard.  Uncorrected  errors  were  nil  with  both 
keyboards. 
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Tha  Alpha-Dot  technique  appears  to  have  potential  as  a  means  to  in¬ 
crease  the  speed,  accuracy,  and  flexibility  of  input  of  battlefield  data 
by  frontline  observers.  Functional  specifications  for  a  militarised  field 
operable  unit  will  be  developed  following  field  tests  of  a  prototype  unit. 


Strub,  M.  H.  Automated  aids  to  on-line  tactical  data  inputting.  Technical 

Paper  262,  February  1975.  (AD  AO 10  350) 

In  an  automated  information  system  the  input  operator  must  rapidly  and 
accurately  convert  free  text  information  into  computer-readable  format 
while  entering  it  into  the  data  base. 

This  report  describes  and  evaluates  a  computer-assisted  message  input¬ 
ting  (CAMI)  aid  designed  to  provide  useful  additional  formatting  instructions 
to  operators. 

Both  a  full  format  and  an  experimental  checklist  format  were  used  with 
the  CAMI  CRT  and  with  a  handbook  of  instructions  and  sample  formats  to  in¬ 
put  scenario-type  messages.  Subjects  were  60  enlisted  men  divided  into  five 
groups,  one  for  each  of  the  experimental  conditions  plus  one  using  only  the 
handbook  and  blank-screen  CRT.  Speed  and  accuracy  were  measured.  Speed  did 
not  differ  significantly  among  the  four  experimental  conditions,  but  the 
fifth  unaided  condition  was  significantly  slower.  No  significant  differences 
in  accuracy  appeared.  However,  over  80%  of  the  total  input  errors  were  dis¬ 
covered  to  be  types  of  error  which  are  not:  detectable  by  present  automated 
error-checking  routines. 


Alderman,  I.  N.  Tactical  data  inputting:  Research  in  operator  performance 
and  training.  Proceedings  of  the  Second  National  Symposium  on  the 
Management  of  Data  Elements  in  Information  Processing.  Washington, 
D.C.:  NBS/ANSI,  1976. 

The  Army  development  of  battlefield  information  systems  established  a 
need  for  research  on  improved  integration  of  human  tasks  and  computer  opera¬ 
tions.  This  paper  reviews  selected  studies  on  the  data  entry  process.  The 
allocation  of  functions/tasks  for  data  entry  and  the  development  of  job  and 
training  aids  are  related  to  human  performance  and  to  the  effectiveness  of 
the  system  as  an  aid  to  the  user. 
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Fields,  A.  F,,  Maisano,  R.  E.,  and  Marshall,  C.  F.  A  comparative  analysis 
of  methods  for  tactical  data  inputting.  Technical  Paper  327,  Septem¬ 
ber  1978.  (AD  A060  562) 


Nearly  all  information  in  tactical  operations  systems  is  input  manu¬ 
ally.  Two  problems  that  arise  when  manually  inputting  data  are  (a)  the: 
introduction  of  errors  in  translating  information  into  computer  format  and 
(b)  the  introduction  of  a  bottleneck  in  total  system  response  time.  There¬ 
fore,  alternative  methods  of  inputting  data  for  accuracy  and  speed  should 
be  evaluated. 
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Four  methods  were  examined  for  speed  and  accuracy  in  inputting  tacti¬ 
cal  messages  concerning  enemy  activity  into  an  Army  computer  format.  The 
methods  were  (a)  typing— the  user  types  the  appropriate  codes  into  a  mes¬ 
sage  format;  (b)  typing  with  an  error  corrector — the  computer  automatically 
attempts  to  correct  common  spelling  and/or  typing  errors;  (c)  menus — the 
user  indicates  which  of  tha  legal  entries  is  desired  from  a  list;  and 
(d)  typing  with  autocompletion  and  an  English  option — the  user  must  type 
only  sufficient  characters  to  uniquely  identify  the  item,  using  either 
the  appropriate  code  or  its  English  definition. 

Thirty-two  enlisted  persons  representative  of  the  class  of  military 
personnel  who  might  serve  as  input  device  operators  input  one  of  four  sets 
of  nine  messages  using  each  inputting  method. 

The  use  of  menus  was  the  most  accurate  inputting  method.  For  users  of 
limited  experience  (1  day  of  inputting) ,  there  were  no  differences  in  speed 
among  the  inputting  methods. 

Consideration  should  be  given  to  the  adoption  of  menus  in  tactical 
operation  systems  with  a  menu  override  option  for  experienced  users.  The 
use  of  an  error  corrector,  autocompletion,  or  an  English  option  is  probably 
not  warranted  unless  operational  use  shows  a  specific  need  for  such  an  aid. 


Nystrom,  C.  O. ,  and  Gividen,  G.  M.  Ease  of  learning  alternative  TOS  message 

reference  codes.  Technical  Paper  326,  September  1978.  (AD  A061  697) 

To  interact  with  a  tactical  data  processing  system  such  as  the  DEVTOS 
or  TOS2,  action  officers  and  user  input-output  device  (U10D)  operators  must 
correctly  enter  a  message  reference  code.  The  senior  author  had  partici¬ 
pated  in  recommending  that  DEVTOS'  letter-letter-number  (LL#)  codes  be  re¬ 
placed  by  LLLL  message  reference  codes  for  the  forthcoming  TOS2  system, 
predicting  in  effect  that  the  LLLL  codes  would  be  easier  tc  learn.  To 
confirm  or  disprove  this  prediction,  40  officers  and  20  enlisted  personnel 
were  employed  to  learn  one  or  the  other  of  the  code  sets,  seated  indi¬ 
vidually  at  a  computer  terminal,  tlu.  i earner  was  shown  message  category 
titles  and  action  names.  The  learner  entered  the  message  reference  code, 
transmitted  it,  ana  received  feedback  as  to  the  correctness  or  incorrect¬ 
ness  of  his  input,  plus  the  correct  code. 

The  error  rate  associated  with  the  learning  of  the  LLLL  codes  was 
less  than  half  that  associated  with  the  learning  of  the  LL#  codes  (13%  ver¬ 
sus  29%  for  EM  working  on  the  G2  message  codes;  11%  versus  21%  for  officers 
on  the  same  G2  message  codes;  and  7%  versus  15%  errors  for  officers  working 
on  G3  message  codes) .  The  LLLL  codes  were  learned  in  about  60%  of  the  total 
time  required  by  the  LL#  codes.  Analysis  of  error  rates  by  character  posi¬ 
tion  revealed  that  the  old  code ' s  numeric  character  and  arbitrary  second 
letter  were  large  sources  of  error.  Several  characters  in  the  LLLL  code 
produced  undesirably  high  error  rates;  the  reasons  for  this  and  possible 
corrections  were  usually  obvious. 
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Gade,  P.  A.,  Fields,  A.  F.,  and  Alderman,  I.  N.  Selective  feedback  as  a 

training  aid  to  on-line  tactical  data  inputting.  Technical  Paper  349, 
November  1978.  (AD  A061  7S9) 

Training  as  Message  Input  Output  Device  (MIOD)  operators  was  given  to 
53  Army  enlisted  personnel  under  one  of  four  training  methods  that  were 
based  on  different  instructional  strategies.  Performance  of  each  partici¬ 
pant  was  assessed  during  training  and  under  one  of  two  operational  configu¬ 
rations  of  the  Tactical  Operations  System  (TOS) . 

Results  showed  that  the  response-sensitive  training  strategy  was  ef¬ 
fective  in  reducing  training  time  without  loss  of  performance  accuracy 
during  transfer- testing.  None  of  the  training  methods  significantly  reduced 
the  amount  of  undetected  errors  that  enter  the  system  through  manual  data 
entry. 


Sidorsky,  R.  C.  Source  data  automation  via  the  Alpha-Dot  tablet:  A  feasi¬ 
bility  study.  Working  Paper  HF-79-07,  June  1979. 

A  study  was  done  to  test  the  feasibility  of  using  a  "tablet"  type  in¬ 
putting  device  in  conjunction  with  the  Alpha-Dot  system  for  data  entry  of 
military  information.  Sixty-six  enlisted  men,  typical  of  the  potential 
users  of  the  device,  participated  in  the  study.  Participants  were  given 
hard-copy,  self-instruction  Alpha-Dot  training  booklets.  After  completion 
of  these,  participants  were  given  test  booklets  on  the  Alpha-Dot  system, 
similar  to  the  training  booklets.  Participants  were  then  instructed  on 
the  use  of  the  v.ablet  device  and  given  a  series  of  Alpha-Dot  characters  to 
input  using  the  device.  Almost  all  participants  showed  positive  transfer 
of  training  from  the  paper  booklet  training  to  the  tablet  device.  Partici¬ 
pants  were  able  to  enter  an  average  of  22.4  characters  per  minute  (cpm) 
during  the  test  booklet  phase.  For  the  tablet  device  the  mean  was  about 
20  cpm.  The  results  indicate  that  with  extended  training,  data  entry  levels 
of  60-75  cpm  could  be  obtained  with  the  tablet  device. 


Mace,  D.  J.,  Harrison,  P.  C. ,  Jr.,  and  Seguin,  E.  L.  Prevention  and  re¬ 
mediation  of  human  input  errors  in  ADP  operations.  Technical  Report 
395,  August  1979.  (AD  A081  730) 

This  report  classifies  types  of  human  errors  made  while  entering  data 
into  automatic  data  processing  (ADP)  systems.  Input  error  causes,  effects, 
and  prevention  are  discussed  specifically  in  terms  of  tactical  ADP  systems 
for  automated  battlefield  information  systems.  The  multi-attribute  utility 
measurement  approach  described  here  provides  a  method  for  evaluating  costs 
and  benefits  of  different  error-reduction  techniques. 


Alderman,  I.  N.,  Ehrenreich,  S.  L.,  and  Bindewald,  R.  Recent  ARI  research 
on  the  data-entry  process  in  Battlefield  Automated  Systems.  Research 
Report  1270,  in  press  (November  1980) . 


This  paper  reviews  ARI  research  designed  to  improve  the  data  entry 
process.  The  first  and  second  sections  of  the  paper  describe  the  data  entry 
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process  in  general  and  in  the  context  of  a  specific  battlefield  automated 
system,  the  Tactical  Operations  System  (TOS) .  Because  it  was  used  as  an 
exemplar  of  the  data  entry  process,  TOS  played  an  important  role  in  t  ie 
development  of  improved  data  entry  procedures.  The  third  section  reviews 
the  findings  and  conclusions  of  the  many  ARI  research  projects  concerned 
with  data  entry.  Among  the  areas  covered  in  ARI's  research  program  are: 


a.  How  to  format  and  display  data  entry  information. 

b.  What  safeguards  can  be  developed  to  reduce  the  number  of  operator 
errors  made  and/or  accepted  by  the  system. 

c.  What  kinds  of  operator  job  aids  can  be  developed  to  improve 
perf ormance . 

d.  How  to  improve  operator  training. 

e.  How  to  make  the  system's  message  codes  easier  to  use  and  more 
memorable . 

f.  How  to  improve  the  design  of  keyboards. 

The  fourth  section  of  the  paper  reports  on  efforts  to  analyze  the  cause  of 
operator  errors.  This  section  also  discusses  the  development  of  a  simula¬ 
tion  of  the  data  entry  process.  The  simulation  is  intended  to  facilitate 
system  design  by  permitting  the  inexpensive  evaluation  of  alternate  data 
entry  procedures.  The  fifth  section  presents  a  general  discussion  of  the 
problems  that  have  been  encountered  by  the  ARX  research  program.  Also  in¬ 
cluded  here  is  a  discussion  on  how  this  program  might  be  improved  in  the 
future.  The  final  section  of  the  paper  summarizes  the  operational  implica¬ 
tions  of  ARI's  research  results. 
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Baker,  J.  D.  Quantitative  modeling  of  human  performance  in  information 

systems.  Ergonomics ,  1970,  13 ,  645-664.  (Also  Technical  Research 

Note  232,  June  1972.)  (AD  746  096) 

This  publication  summarizes  an  approach  toward  developing  a  general 
information  system  model  which  focuses  on  man  and  considers  the  computer 
only  as  a  tool.  The  ultimate  objective  is  to  produce  a  simulator  which 
will  yield  measures  of  system  performance  under  different  mixes  of  equip¬ 
ment,  personnel,  and  procedures.  The  model  is  structured  around  three 
dimensions:  (1)  data  flow  and  processing,  (2)  task  analysis,  and  (3)  out¬ 

side  sources  of  variation  in  the  system.  Five  basic  functional  areas 
represented  in  the  model  are  screening  incoming  messages,  transforming  mes¬ 
sages  to  computer  format,  inputting  messages,  assimilating  data  displayed, 
and  deciding  on  courses  of  action  based  on  information  obtained.  The  im¬ 
pact  of  human  performance  in  the  model  is  reflected  in  measures  of  system 
performance  (thoroughness  of  information  and  responsiveness  to  the  user 
need)  and  quality  of  information  (completeness  and  accuracy) . 


Siegel,  A.  1.,  Wolf,  J.  J.,  and  Leahy,  W.  R.  A  digital  simulation  model 

of  message  handling  in  the  Tactical  Operations  System:  I.  The  model, 
its  sensitivity,  and  user's  manual.  Research  Memorandum  73-5,  Novem¬ 
ber  1973.  (Also  Technical  Report  TR-77-A-23,  October  1977.) 

(AD  A047  104) 

The  results  of  research  into  implementation  of  a  quantitative  model 
of  human  performance  in  information  systems  are  presented.  The  principal 
efforts  were  to:  (1)  define  the  most  influential  psychosocial  variables 
inherent  in  the  mission  of  interests,  (2)  incorporate  these  into  a  logic 
for  a  digital  simulation  model,  and  (3)  develop  a  computer  program  reflect¬ 
ing  this  model.  The  Tactical  Operations  System  (TOS)  for  which  the  model 
was  developed  is  described,  and  model  features  involving  predictive  capa¬ 
bility  and  system  effectiveness  measurements  are  presented.  The  results 
of  series  of  model  sensitivity  tests  under  a  variety  of  parametric  input 
conditions  cure  reported. 


Leahy,  W.  R. ,  Lautman,  M.  R. ,  Bearde,  J.  L. ,  and  Siegel,  A.  I.  A  digital 

simulation  model  of  message  handling  in  the  Tactical  Operations  System: 
III.  Further  extensions  of  the  model  for  increased  interaction.  Re¬ 
search  Memorandum  74-11,  August  1974,  (Also  Technical  Report  TR-77- 
A25,  October  1977.)  (AD  A047  105) 

The  present  report  is  the  last  in  a  series  of  three  which  describe  the 
inner  working  of  MANMODEL,  a  simulation  of  the  U.S.  Army  Tactical  Operations 
System  (TOS) .  MANMODEL  yields  measures  of  system  performance  under  differ¬ 
ent  mixes  of  equipment,  personnel,  and  procedures.  Sensitivity  tests  of 
MANMODEL  have  been  successfully  completed  and  are  reported  here.  Formal 
validation  of  MANMODEL  as  a  predictor  of  performance  is  discussed  as  a  logi¬ 
cal  next  step  in  its  evolution. 
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Siegel,  A.  1.,  Wolf,  J.  J. ,  Leahy,  W.  R. ,  and  Bearde,  J.  L.  A  digital 

simulation  model  of  message  handling  in  the  Tactical  Operations  Sys¬ 
tem:  II.  Extensions  of  the  model  for  interactivity  with  subjects 
and  experimenters.  Research  Memorandum  73-6,  December  1973.  (Also 
Technical  Report  TR-77-A24,  October  1977.)  (AD  A046  407) 

Extensions  and  improvements  are  described  to  a  previously  developed 
digital  computer  model  for  simulating  the  actions  of  operational  field  Army 
personnel  performing  their  message  processing  tasks  during  a  Tactical  Opera¬ 
tions  Systems  (TOS)  mission.  The  computer  model  was  made  interactive  via 
cathode  ray  tube  to:  (1)  enable  an  experimenter  to  initiate  and  control 
computer  simulation  runs,  and  (2)  allow  TOS  operators  at  a  computer  terminal 
to  perform  selected  tasks  during  the  simulation.  A  series  of  model  improve¬ 
ments  found  desirable,  as  a  result  of  prior  simulation  run  experience,  is 
described. 

A  revised  version  of  the  "user's  manual"  for  the  model  is  presented 
along  with  an  Interactive  Model  User's  Manual. 


Leahy,  w.  R. ,  Siegel,  A.  I.,  and  Wolf,  J.  J.  A  digital  simulation  model 
of  message  handling  in  the  Tactical  Operations  System:  IV.  Model 
integration  with  CASE  and  SAMTOS.  Technical  Report  413 ,  October 
1979.  (AD  A086  4!>j) 

The  human  performance  oriented  computer  simulation,  called  MANMOD,  of 
the  u.s.  Army's  Tactical  Operations  System  (TOS),  was  modified  and  extended 
to  allow  increased  capability  and  generality.  The  modifications  and  exten¬ 
sions  included  but  were  not  limited  to:  (1)  incorporation  of  the  capability 
to  simulate  error  message  receipt  and  processing,  (2)  interaction  and  inte¬ 
gration  with  a  modified  CASE  model  and  with  the  SAMTOS  model  which  princi¬ 
pally  simulate  TOS  equipment  functions,  and  (3)  implementation  of  the  MANMOD 
on  the  Univac  1108  computer  system.  The  modified  MANMOD  was  tested  relative 
,  to  sensitivity  and  reasonableness  of  output.  The  evidence  supports  the  use 
of  the  model  for  a  number  of  functions  relative  to  system  design,  training 
requirements  and  objectives  derivation,  personnel  requirements,  and  tradeoffs. 


Leahy,  W.  R. ,  Siegel,  A.  I.,  and  Wolf,  J.  J.  A  digital  simulation  model 
of  message  handling  in  the  Tactical  Operations  System:  V.  User's 
guide  to  the  integrated  MANMOD/CASE/SAMTOS  computer  simulation. 
Technical  Report  414,  October  1979.  (AD  A086  355) 

This  document  is  a  complete  juide  for  the  use  and  interpretation  of  the 
computer  simulation  program — MANMOD.  The  program  is  stored  in  files  on  the 
Univac  1108  computer  at  the  Edgewood  Arsenal.  The  MANMOD  program  calls  the 
Army  Research  Institute's  MANMOD  simulation  of  the  Tactical  Operations  Sys¬ 
tem  and  provides  an  interface  with  the  CASE  and  the  SAMTOS  computer  simula¬ 
tion  models. 

The  MANMOD  program  is  written  in  FORTRAN  IV.  Sufficient  information 
is  provided  in  this  manual  to  allow  simulations  to  be  performed  by  indi¬ 
viduals  with  minimum  computer-related  experience.  In  addition,  detailed 
flow  charts  and  variable  lists  are  provided  for  the  use  of  skilled 
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programmers  who  desire  a  more  technical  description  of  the  mechanics  of  the 
simulation  and  to  allow  program  changes  to  be  made  more  easily. 


Baker,  J.  D.  Modeling  the  user.  Working  Paper  HF-80-3 ,  November  1979. 

This  report  provides  an  overview  of  two  simulation  models  concerned 
with  different  aspects  of  the  impact  of  man  on  system  effectiveness.  The 
first,  MANMODEL,  was  developed  as  a  model  of  message  flow  through  the  Tacti¬ 
cal  Operations  System  (TOS)  from  receipt  of  free-text  messages  at  the  opera¬ 
tions  center  to  computer  storaqe  of  the  message.  The  second  model,  SIMTOS, 
is  a  man-in-the-loop  simulation  developed  to  support  research  on  staff  offi¬ 
cer  performance.  This  overview  was  originally  prepared  for  presentation  at 
the  NATO  Advanced  Study  Institute  on  Man  Computer  Interactions,  MATI,  Greece 
5-18  September  1976. 
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Patten,  S.  M.  An  inductive  taxonomy  of  combat  intelligence  data.  Research 
Memorandum  74-14,  December  1974.  (AD  A076  802) 

To  effectively  handle  an  increased  volume  of  data,  an  intelligence  ana¬ 
lyst  working  on  a  specific  problem  must  be  able  to  selectively  obtain  infor¬ 
mation  relevant  to  the  problem  without  be-oming  bogged  down  in  irrelevant 
information.  To  achieve  an  efficient  information  retrieval  capability, 
whether  using  a  manual  or  automated  procedure,  it  is  desirable  for  data  to 
be  stored  in  a  form  compatible  with  the  requirements  of  the  analyst. 

This  report  presents  an  outline  of  one  approach  to  data  organization 
and  provides  the  information  taxonomy  produced  when  this  methodology  was 
applied  to  one  message  set.  While  a  number  of  information  categories  were 
unique  to  this  message  set,  the  taxonomy  is  expected  to  constitute  a  valu¬ 
able  starting  point  for  future  efforts  of  this  type. 


Siegel,  A.  I.,  Pfeiffer,  M.  G.,  and  Warms,  T.  M.  Development  of  evaluation 

of  a  content  analytic  approach  in  Army  field  system  data  organization. 

Technical  Report  TR-77-A11,  August  1977.  (AD  A042  075) 

This  report  describes  the  logic,  methods,  and  results  of  a  study  into 
the  derivation  of  a  content  analytic  approach  to  Army  field  system  data  or¬ 
ganization.  First  was  a  linguistic  analysis  of  a  set  of  battlefield  messages 
and  several  multidimensional  scaling  analyses  of  a  sample  of  messages  repre¬ 
senting  all  information  classes  yielded  by  the  linguistic  analysis.  Four 
multidimensional  scaling  analyses  were  completed.  Each  was  based  on  the 
linguistic  similarity  perceptions  of  one  of  four  experienced  battlefield 
analysts.  High  agreement  was  found  among  the  factorial  structures  yielded 
by  the  data  of  each  battlefield  analyst. 

Accordingly,  an  overall  analysis  was  completed,  and  15  factors  were 
identified  as  representing  the  perceptual  substrate  of  the  Army  field  in¬ 
formation  linguistic  system.  On  the  basis  of  the  derived  factors,  a  battle¬ 
field  language  taxonomy  was  developed.  The  taxonomy  was  tested  in  two  sep¬ 
arate  field  oriented  experiments . 

The  results  of  these  experiments  indicated  that  intelligence  analysts 
cam  classify  messages  reliably  within  the  taxonomy  and  that  they  can  reliably 
use  the  taxonomy  for  inquiry  purposes.  Moreover,  adequate  subjective  confi¬ 
dence  was  expressed  by  the  analysts  in  the  use  of  the  system.  Finally,  a 
computer  system  for  automatic  classification  of  battlefield  messages  is 
presented. 


Miron,  M.  S.,  Patten,  S.  M. ,  and  Halpin,  S.  M.  Content  analysis  and  the  or¬ 
ganization  of  combat  intelligence  data.  Research  Memorandum  78-7, 
February  1978.  (AD  A077  956) 

This  research  examined  and  tested  an  application  of  a  high-speed  data 
processing  technique  designed  to  automatically  provide  organizational 
structure  for  incoming  intelligence  reports.  A  system  of  computer  routines 
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known  as  the  General  Inquirer  was  developed  to  analyze  message  content.  The 
computer  identifies  and  catalogs  a  set  of  previously  selected  words  and 
phrases  in  the  text  of  intelligence  reports  from  the  field. 

The  data  base  consisted  of  33  intelligence  reports  recorded  by  the  28th 
Infantry  Division  just  before  the  Battle  of  the  Bulge  and  7  false  reports 
designed  to  test  the  capabilities  of  the  system. 

The  sample  of  40  intelligence  reports  was  keypunched  on  data  entry 
cards,  a.  dictionary  of  critical  concepts  was  constructed,  rules  for  identi¬ 
fying  these  concepts  as  they  occurred  in  the  text  were  developed,  the  oc¬ 
currences  of  the  identified  concepts  were  tabulated  for  each  message,  and 
correlations  and  factor  analyses  were  computed  using  these  tabulations  of 
identified  concepts. 

It  was  found  that  this  logically  coherent  structure  could  aid  intelli¬ 
gence  analysts  in  organizing  and  analyzing  the  data  in  the  message  set. 
However,  the  content-analytic  procedures  need  to  be  refined.  Two  inade¬ 
quacies  were  the  dictionary  definition  language  and  the  fact  that  the  en¬ 
tire  General  Inquirer  System  was  programed  only  for  the  IBM  S/360  or  S/370 
computer.  Nonetheless,  sufficient  promise  existed  to  warrant  further 
investigation . 
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Operating  Systems,  Inc.  MIQSTURE;  An  experimental  on-line  language  for 

Army  tactical  intelligence  information  processing.  Technical  Report 

TR-78-A25 ,  July  1978.  (AD  A064  323) 

Analysis  and  specifications  are  presented  for  applying  a  mixed-initiative 
query  system  to  intelligence  data  processing.  The  project  had  four  principal 
steps:  (a)  development  of  a  statement  of  requirements  for  interactive  data 

processing  in  tactical  intelligence  analysis;  (b)  extension  and  application 
to  the  requirements  identified  in  the  first  step  of  a  concept  of  a  mixed- 
initiative  system,  focused  on  task  requirements  and  user  needs;  (c)  develop¬ 
ment  of  detailed  design  specifications  for  six  selected  subsets  of  MIQSTURE 
(the  Mixed  Initiative  Query  System  with  Task  and  User  Related  Elements) ;  and 

(d)  preliminary  evaluation  of  MIQSTURE.  The  six  subsets  developed  in  some 
detail  were  (a)  normal  querying  of  message  records;  (b)  automatic  alerting 
and  input-driven  querying;  (c)  normal  tabular  data  base  querying  and  calcu¬ 
lation;  (d)  aids  for  representing  the  tasks  which  form  the  context  of  MIQSTURE 

(e)  procedures  for  cross-referencing  graphic  symbols  and  data-base  elements; 
and  (f)  defining  and  maintaining  data  file  structures.  An  analytic  evaluation 
of  MIQSTURE  indicated  high  potential  for  application  to  tactical  intelligence 
processing  and  other  data  processing  situations  involving  a  relatively  re¬ 
stricted  range  of  well-defined  tasks . 


Nawrocki,  L.  H.  Word  abbreviations  in  man-computer  communication  systems. 

Working  Paper  HF-^q-CK,  June  1979. 

This  experiment  is  the  first  stage  of  an  effort  to  empirically  evaluate 
the  feasibility  and  utility  of  incorporating  natural  linguistic  processes 
into  the  word  abbreviations  rules  for  automated  systems.  The  purpose  of  the 
present  experiment  was  to  compare  decoding  performance  using  human  generated 
abbreviations  lists  to  performance  using  lists  stressing  primarily  economy, 
consistency,  or  both. 

Each  of  75  U.S.  Army  enlisted  personnel  were  shown  one  of  five  sets  of 
120  single  word  abbreviations  over  a  cathode  ray  tube  (CRT) .  Each  set  of 
abbreviations  was  generated  by  one  of  five  different  sets  of  rules  for  the 
formation  of  abbreviations.  Their  task  was  to  enter  their  best  guess  at  the 
source  word  for  each  abbreviation  onto  the  CRT.  They  were  also  asked  to  in¬ 
dicate,  on  a  0-100  scale,  hov  confident  they  were  that  their  responses  on 
each  abbreviation  were  correct. 

Results  indicate  that  economy  appears  to  be  the  primary  dimension  of 
abbreviations  affecting  the  research.  The  relationship  between  economy  and 
percentage  of  abbreviations  identified  correctly  was  an  inverse  one,  however. 
The  lower  the  economy  of  the  list,  the  higher  the  percentage  the  number  of 
source  words  identified  correctly.  Results  for  all  five  sets  of  abbrevia¬ 
tions  are  given  and  discussed. 
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Potash,  L.  M.  Effects  of  retrieval  term  specificity  on  information  retrieval 

from  computer-based  intelligence  systems.  Technical  Paper  379,  July 

1979.  (AD  A072  312) 

This  experiment  assessed  the  impact  of  using  two  levels  of  retrieval 
terms  on  formulating  and  inputting  query  statements.  One  level  used  specific 
terms  to  retrieve  one  element  of  information*  the  other  level  used  global 
terms  to  retrieve  blocks  of  information  that  would  otherwise  require  five  to 
six  specific  terms. 

Participants  in  the  study  were  enlisted  personnel  from  Fort  Belvoir, 

Va. ,  with  General  Technical  (GT)  scores  2110.  Participants  were  assigned 
either  to  a  specific  group  allowed  to  use  only  specific  retrieval  terms,  or 
to  a  global-specific  group  allowed  to  use  both  global  and  specific  retrieval 
terms.  The  retrieval  terms  and  their  definitions  were  listed  in  a  data  dic¬ 
tionary  and  were  displayed  on  a  data  name  chart. 

After  a  5-minute  typing  test,  participants  were  given  instructions  on 
how  to  write  query  statements  in  the  GIM  II  Query  Language  used  in  Army  Sys¬ 
tem  for  Standard  Support  Terminals  (ASSIST) . 

Participants  were  then  given  a  set  of  48  problems.  For  each  problem, 
the  participants  had  to  write  and  type  a  query  statement  that  would  satisfy 
the  information  requirements.  An  electric  typewriter  simulated  the  keyboard 
input  of  a  computer  terminal.  Using  a  stopwatch,  each  participant  recorded 
the  time  it  took  to  write  and  type  query  statements.  After  finishing  the 
problems,  each  participant  was  given  an  ancillary  learning  test  to  assess  how 
well  specific  terms  had  been  learned.  Finally,  participants  rated  the  ease 
of  use  of  the  query  language,  indicated  how  they  went  about  writing  query 
statements,  and  rated  the  value  of  using  global  terms. 

The  opportunity  to  use  global  terms  had  no  effect  either  rn  the  time 
needed  to  write  query  statements  or  on  the  accuracy  of  typed  query  statements. 
Where  the  use  of  global  terms  was  applicable,  substantial  savings  in  the  time 
required  to  input  query  statements  was  shown.  With  the  exception  that  the 
globa_-specific  group  reported  that  it  made  more  use  of  the  data  name  chart 
before  using  the  data  dictionary,  the  two  groups  indicated  that  they  went 
about  writing  query  statements  in  approximately  the  same  way.  Both  groups 
gave  high  ratings  to  the  value  of  using  global  terms. 

Use  of  global  terms  is  not  recommended  unless  the  s.  ->ecific  items  of  in¬ 
formation  subsumed  under  the  global  term  are  normally  retrieved  together 
frequently . 


Moses,  F.  L, ,  and  Potash,  L.  M.  Assessment  of  abbreviation  methods  for  auto¬ 
mated  tactical  systems.  Technical  Report  398,  August  1979.  (AD  A077  840) 

Alternative  methods  of  producing  abbreviations  of  single  words  for  use 
in  automated  military  systems  were  empirically  evaluated.  Each  of  50  en¬ 
listed  men  performed  three  tasks  designt-1  to  assess  five  different  abbrevia¬ 
tion  methods  including  two  versions  of  truncation,  two  versions  of  contrac¬ 
tion,  and  one  procedure  based  on  current  Army  practice.  In  the  first  four 
methods,  abbreviation  length  was  directly  related  to  word  length.  In  Task  A, 
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participants  used  a  10-point  scale  to  rate  preference  for  abbreviations  for 
each  of  60  terms.  Thirty  of  these  terms  were  repeated  in  Task  B>  the  other 
30 ,  in  Task  C.  Task  B  required  participants  to  decode  60  abbreviations 
into  their  original  terms.  Finally,  Task  C  asked  participants  to  encode 
(i.e.,  generate)  a  meaningful  abbreviation  for  each  of  60  terms.  Results 
showed  that  abbreviations  produced  using  simple  truncation  were  consistently 
preferred,  easily  decoded,  and  frequently  used  for  encoding  terms.  Scores 
for  abbreviations  from  current  Army  practice  were  among  the  lowest  except 
in  Task  C  (encoding) .  Experience  with  abbreviations  in  Task  A  (preference) 
made  decoding  of  identical  abbreviations  significantly  easier  in  Task  B, 
but  did  not  affect  the  abbreviations  encoded  by  participants  in  Task  c. 
Simple  truncation  is  the  method  suggested  to  produce  good  single  word  ab¬ 
breviations  quickly  and  to  reduce  the  errors  and  the  time  requirements  for 
user  interactions  with  battlefield  automated  systems. 


Katter,  R.  V.,  and  Bell,  G.  Experimental  evaluation  of  concepts  for  MIQSTURE s 
An  online  literature  language  for  tactical  intelligence  processing. 
■>;earch  Note  80-19.  December  1979. 

This  project  evaluated  selected  aspects  of  an  online  language  for  Army 
tactical  intelligence  processing,  called  MIQSTURE.  In  the  first  of  two  ex¬ 
periments,  U.S.  Army  tactical  intelligence  officers  provided  expert  judgments 
on  work-related  and  information  utilisation  aspects  of  descriptions  of  se¬ 
lected  tasks  from  Army  tactical  intelligence  processing.  The  results  pro¬ 
vided  indications  of  what  query  methods  have  potentials  as  aids  for  intelli¬ 
gence  analysts.  In  the  second  experiment,  an  evaluation  was  made  of  the  ef¬ 
ficacy  of  a  familiarisation/refresher  display  arrangement  for  developing  and 
maintaining  a  useful  level  of  user/operator  familiarity  with  little-used  but 
essential  elements  of  the  interactive  language.  The  results  were  promising 
for  the  display  arrangement. 


Ehrenreich,  S.  L.  Design  recommendations  for  query  languages.  Technical 
Report  484,  November  1980. 

For  a  discussion  of  this  report,  see  the  previous  citation  on  p.  11-21 
of  this  paner. 
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McKendry,  J.  M. ,  Macs,  D.  J. ,  and  Baker,  J.  D.  Implications  of  BESRL  re¬ 
search  for  displays  in  tactical  information  processing.  Technical 
Research  Report  1156,  January  1969.  (AD  6B8  581) 

The  present  report  summarizes  BESRL  (now  ARI)  display  research  findings 
that  are  amenable  to  immediate  or  near-future  application  to  operational 
human  factor  problems  in  the  display  and  assimilation  of  information  in  field 
facilities. 

Problem  areas  identified  through  field  exercise  observations  are  re¬ 
viewed  in  the  context  of  greatest  applicability  of  laboratory  research  done 
by  BESRL  as  well  as  the  feasibility  of  procedural  modifications  indicated 
by  the  research  findings.  Major  principles  pertaining  to  specific  opera¬ 
tional  needs  are  defined.  These  are  then  translated  into  suggestions  and 
their  application  illustrated  by  specific  examples  (especially  with  refer¬ 
ence  to  the  potential  of  TOTE-  'type  displays  and  simple  coding  techniques  for 
facilitating  information  presentation  and  assimilation) .  Research  needs 
bearing  on  human  factor  problems  involved  in  decision-making  tasks  in  the 
operational  setting  are  also  discussed. 


Nawrocki,  L.  H.  Alpha-numeric  versus  graphic  displays  in  a  problem-solving 

task.  Technical  Research  Note  227,  September  1972.  (AD  748  799) 

The  experiment  reported  here  was  designed  to  determine  how  alpha-numeric 
and  graphic  presentation  of  computer  data  displays  affect  performance,  in 
terms  of  speed  and  accuracy,  under  two  sets  of  system  requirements x  (1)  the 
need  to  base  a  decision  on  memory  of  information  previously  displayed  versus 
no  memory  requirement,  and  (2,  the  complexity  of  information  to  be  held  in 
memory  (memory  load) . 

Results  of  this  study,  in  conjunction  with  previous  comparative  evalu¬ 
ation  of  the  two  alternative  display  modes,  suggest  that  under  a 'variety  of 
tasks  and  conditions,  there  is  no  clear-cut  advantage  to  the  use*of  either 
alpha-numeric  or  graphic  displays  when  memory  of  displayed  material  was  re¬ 
quired,  Hence,  the  choice  of  display  type  may  be  primarily  one  of  cost 
consideration  if  time  and  accuracy  are  the  primary  determinants  of  system 
performance.  However,  when  memory  was  not  required,  alpha-numeric  displays 
resulted  in  fewer  errors  of  omission  than  did  graphic  displays.  It  was 
also  found  that  increasing  complexity  caused  a  deterioration  in  speed  when 
no  memory  was  required  and  a  decrement  in  accuracy  when  memory  was  required. 


Nawrocki,  L.  H.  Graphic  versus  tote  display  of  information  in  a  Simulated 
Tactical  Operations  System.  Technical  Research  Note  243,  June  1973. 
(AD  766  217) 

As  part  of  a  continuing  study,  ARI  compared  the  effects  of  graphic  and 
tote  display  modes  on  the  speed  and  quality  of  decisions  based  on  the  dis¬ 
played  information. 
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In  the  context  of  ARl's  simulated  Tactical  Operations  System  (SIMT03) , 
28  officers,  divided  into  two  groups,  were  presented  a  command  decision 
task.  For  one  group,  information  requested  by  the  officers  from  the  data 
base  was  presented  in  graphic  format  emphasising  spatial  and  symbolic  coding 
of  the  data.  For  the  other  group,  the  data  were  presented  in  tote  form  em¬ 
phasising  tabular  arrangement  and  alpha-numeric  coding.  Evaluation  of  the 
two  display  modes  was  in  terms  of  speed  and  accuracy  of  the  final  decision 
And  the  number  of  nenredundant  facts  accessed  by  the  subject. 

The  two  groups  did  not  differ  significantly  in  accuracy  of  decision, 
speed  of  decision,  or  number  of  facts  accessed.  Therefore,  the  more  eco¬ 
nomical  tote  form  of  displays  should  be  used  wherever  feasible.  Where 
graphic  displays  are  used,  the  complexity  and  density  of  information  pre¬ 
sented  can  be  considerably  reduced  by  conversion  to  a  form  suitable  for  CRT 
or  printout. 


Colson,  K.  R. ,  Freeman,  F.  S.,  Mathews,  L.  P.,  and  Stettler,  J.  A.  Devel¬ 
opment  of  an  informational  taxonomy  of  visual  displays  for  Army  tacti¬ 
cal  data  systems.  Research  Memorandum  74-4,  February  1974. 

(AD  A082  951) 

This  is  a  detailed  account  of  research  conducted  to  develop  a  taxonomy 
of  visual  displays  for  the  Army's  tactical  data  systems.  A  major  problem 
in  effective  display  system  design  and  development  has  been  the  lack  of  a 
comprehensive  frame  of  reference  and  associated  terminology  that  can  be  used 
to  characterize  displays  and  the  functions  they  perform  as  part  of  a  large 
information  processing  system.  The  taxonomy  provides  a  structure  for  defin¬ 
ing,  evaluating,  and  comparing  displays  based  on  the  user's  information  needs. 
An  attempt  was  made  to  evaluate  the  taxonomy  using  operational  display  sys¬ 
tems.  The  report  includes  background  and  references. 


Granda,  T.  M.  A  comparison  between  a  standard  map  and  a  reduced  detail  map 
within  a  Simulated  Tactical  Operations  System  (SIMTOS) .  Technical  Paper 
274,  June  1976.  (AD  A028  752) 

The  use  of  reduced  maps  in  place  of  standard  Army  maps  has  often  been 
recommended  for  the  upper  command  (e.g.,  Division)  levels.  Within  the  con¬ 
text  of  a  simulated  tactical  operations  system,  several  variables  that  were 
thought  roost  likely  to  be  affected  by  the  use  of  reduced  detail  maps  or 
echelon  information  needs  were  observed  in  an  offensive  planning  phase  and 
an  offensive  combat  phase.  Among  the  dependent  variables  included  were 
number  of  data  frames  accessed,  time  to  access  data  frames,  and  combat 
effectiveness  scores. 

The  analyses  of  variance  performed  on  the  variables  show  significant 
main  effects  for  the  echelon  factor  for  the  amount  and  type  of  information 
accessed,  and  for  one  combat  effectiveness  score.  No  significant  differ¬ 
ence  in  performance  during  planning  or  combat  was  associated  with  the  level 
of  map  detail  factor.  Therefore,  the  feasibility  of  using  reduced  detail 
maps  in  conjunction  with  a  computerized  graphic  display  capability  should 
be  investigated. 
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Grands,  T.  M.  An  avaluation  of  visual  search  behavior  on  a  cathode  ray  tube 
utilising  the  window  technique.  Technical  Paper  263.  February  1978, 

(AD  A053  352) 

Large  or  dense  information  displays  cannot  always  be  presented  legibly 
on  a  cathode  ray  tube  (CRT)  all  at  once.  This  experiment  evaluated  the 
feasibility  of  using  a  window  technique  in  CRT  visual  search.  The  first 
part  of  the  experiment  considered  whether  the  super-display  should  be  static 
and  the  window  should  move  or  whether  the  super-display  should  move  and  the 
window  should  be  static.  The  second  part  of  the  experiment  considered 
whether  the  length  of  a  memorised  target  set  affected  information  processing. 

Ten  participants  searched  a  aeries  of  alphanumeric  CRT  displays  for 
specific  information  targets.  Display  motion  control,  length  of  target  list, 
target  density,  nontarget  density,  and  window  size  were  independent  variables? 
time  effects  on  performance  were  also  measured.  Five  participants  controlled 
window  motion  and  five  controlled  super-display  motion. 

Results  showed  that  a  window  technique  is  feasible  for  searching  for 
targets  on  a  CRT.  Participants  found  most  of  the  targets  and  made  few  move¬ 
ment  errors.  The  most  important  variables  were  target  density  and  the 
display-motion  relationship.  Participants  using  a  moving  window  found  more 
targets  than  those  using  the  moving  super-display;  however,  the  moving  super¬ 
display  group  made  fewer  control  errors  and  took  less  time  to  view  the  en¬ 
tire  display.  Participants  searched  the  displays  as  quickly  for  seven  tar¬ 
gets  as  they  did  for  five  or  three  targets.  Window  technique  results  can 
apply  to  any  task  in  which  the  entire  display  cannot  be  presented  at  one 
time;  information  processing  results  can  help  structure  similar  tasks. 


Moses,  F.  L. ,  and  Maisano,  R,  E.  User  performance  under  several  automated 

approaches  to  changing  displayed  maps.  Technical  Paper  366,  June 

1979.  (AD  A073  726) 

Performance  in  a  route  selection  task  was  used  to  evaluate  automated 
methods  for  changing  from  one  displayed  segment  of  a  map  to  another.  Par¬ 
ticipants  were  24  Army  officers  who  chose  successive  6.75  x  9  km  map  seg¬ 
ments  (1:50,000  scale)  fer  solving  12  problems.  Each  problem  requested  the 
fastest  road  route  between  pairs  of  cities  within  a  60  x  81  km  region. 
Participants  solved  problems  by  electronically  marking  road  routes  across 
map  segments.  Methods  for  changing  map  segments  were  (a)  continuous  map 
scanning  and  (b)  discrete  map  segments  using  three  different  amounts  of 
border  overlap  (0%,  25%,  and  50%).  Results  showed  that  different  map  change 
conditions  did  not  significantly  affect  the  quality  of  routes  chosen.  The 
least  time  for  problem  solutions  occurred  when  map  segments  with  50%  overlap 
were  used,  although  25%  overlap  produced  similar/  data.  Designers  of  map 
display  systems  for  the  military  could  optimize, user  performance  time  with 
discrete  map  segments  that  overlap  by  about  25*y'. 
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Gellman,  L.  H.  Recent  AR1  research  on  alternative  display  formats.  Re¬ 
search  Report  1294,  November  1930. 

The  need  for  alternative  display  formats  results  from  the  inadequacy 
of  present  symbologies  for  automated  displays. 

The  paper  is  divided  into  three  sections .  Part  I  discusses  the  prob¬ 
lems  associated  with  clutter  and  how  alternative  formatting  can  lessen 
these  problems.  Part  II  examines  two  formatting  techniques  (sequencing 
and  coding)  and  relevant  ARI  research.  Finally  Part  III  presents  some  sug¬ 
gestions  for  implementing  these  research  findings. 


Sidorsky,  R.  C.  Color  coding  in  tactical  displays:  Help  or  hindrance. 
Technical  Report  496,  November  1980. 

The  purpose  of  this  research  was  to  determine  the  effectiveness  of  color 
coding  as  a  means  for  making  tactical  displays  easier  and  faster  to  analyze. 


Four  groups  of  analysts  observed  visual  displays  of' symbolically  encoded 
information  depicting  the  type,  status,  and  disposition  of  friendly /enemy 
units  on  the  battlefield.  Each  group  used  a  different  code  format.  The  code 
formats  differed  with  respect  to  the  category  of  information  conveyed  by 
color  coding.  The  performance  of  all  groups  was  measured  on  three  tasks. 

The  tasks  differed  with  regard  to  their  level  of  complexity,  i.e.,  required 
the  extraction  of  one,  two,  or  three  categories  of  information  from  each 
symbol. 

Analyses  of  variance  were  performed  on  measures  of  speed  and  accuracy 
information  processing.  The  results  indicate  that  color  can  be  of  sub- 
‘  ant.ial  benefit  in  terms  of  reduced  processing  time  and  errors.  Information 
portrayed  by  traditional  shape  coding  requires  as  much  as  75%  more  process¬ 
ing  time  and  produces  200%  to  800%  more  errors.  However,  color  is  of  value 
only  if  it  is  used  to  encode  information  that  is  extracted  at  the  first  level 
of  analysis.  The  establishment  of  a  standardized  sequence  of  colors  for 
tactical  symbology  to  portray  the  status  or  condition  of  tactically  relevant 
objects  or  events  is  suggested. 
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Bersh,  P. ,  Moses,  F.  L.,  and  Maisano,  R.  E.  Investigation  of  the  strength 

of  association  between  graphic  symbology  and  military  information. 

Technical  Paper  324,  September  1978.  (AD  A064  260) 

This  investigation  sought  to  determine  the  strength  of  associations  of 
simple  codes  or  geometrical  shapes  with  military  concepts.  Symbols  were 
chosen  for  easy  generation  on  computer  displays  or  for  ease  of  manual 
reproduction. 

Each  of  114  enlisted  men  ranked  battlefield  information  categories 
(e.g.,  Unit  Level,  Danger,  and  fir* power)  in  the  order  of  their  strength  of 
association  with  different  sets  of  symbols,  with  numbers  of  each  set  vary¬ 
ing  in  a  single  characteristic.  In  a  related  task,  these  persons  were  asked 
to  rank  order  military  branch  or  duty  designations  and  tactical  function 
terms  by  how  well  they  were  suggested  by  both  geometrical  symbols  and  mis¬ 
cellaneous  "stick"  symbols.  A  second  group  of  137  enlisted  persons  had  the 
reverse  task  of  ranking  symbols  against  one  another  for  their  strength  of 
association  with  military  concepts.  Both  groups  of  participants  had  only 
limited  prior  exposure  to  military  symbology. 

Results  suggest  that  "natural"  associations  can  be  found  between 
graphic  codes  or  symbols  and  military  concepts.  Results  were  categorized 
as  high,  moderate,  minimal,  and  insignificant  associations,  based  on  the 
statistical  significance  of  differences  among  ranks  and  on  the  degree  of 
reflexive  associations  between  symbols  and  concepts.  About  half  of  the 
primary  matches  between  tactical  concepts  and  symbols  fell  into  high-  and 
moderate-association  categories.  These  matches  included  the  currently  used 
link  between  numerosity  and  Unit  Level  (e.g.,  company  and  division)  as  well 
as  the  link  of  color  with  Danger  and  of  square  with  Service  Support.  The 
remaining  association  results,  including  the  currently  used  associations 
of  an  ellipse  with  Armor,  X  with  Infantry,  and  of  color  for  Friend/Enemy, 
fell  into  the  minimal-  or  insignificant-association  categories.  The  data 
gathered  identify  symbol  characteristics  for  consideration  in  modifying 
current  symbology.  Natural  codes  or  symbols  based  on  such  a  consensus 
should  be  easily  learned  and  interpreted. 


Hemingway,  P.  W. ,  Kubala,  A.  L. ,  and  Chastain,  G.  D.  Study  of  symbology 

for  automated  graphic  displays.  Technical  Report  TR-79-A18,  May  1979. 
(AD  A076  916)  "  '  ’  . . 

An  extensive  literature  survey  indicated  a  great  variety  of  available 
alternative  symbology  formats.  Before  a  specific  format  is  selected,  the 
necessary  information  requirements  for  a  particular  position  must  be  deter¬ 
mined.  For  Army  tactical  displays,  this  information  should  be  gathered  in 
experiments  using  realistic  battle  scenarios.  As  a  related  effort,  alter¬ 
native  symbol  formats  were  evaluated  by  civilians.  Naive  subjects  generally 
preferred  a  pictorial  symbol  over  traditional  military  alternatives .  The 
report  is  written  for  a  military  audience. 


Symbology 


Sidorsky,  R. ,  Gellman,  L.  H.,  and  Moses,  F.  L.  Survey  of  current  develop¬ 
ments  in  tactical  symbology:  Status  and  critical  issues.  Working 
Paper  HF-79-03,  May  1979. 

This  report  presents  the  results  of  a  structured  survey  of  nine  of 
the  Army's  research  and  development  efforts  related  to  symbology.  This  sur¬ 
vey  dealt  with  such  factors  as  mission,  tasking,  and  coordination  of  the 
agency  as  well  as  methodology,  equipment,  perceived  current  problems,  and 
future  plans  with  regard  to  symbology.  Three  general  problems  in  symbology 
briefly  are  discussed:  the  lack  of  an  adequate  definition  of  symbology, 
the  lack  of  a  symbology  typology,  and  the  need  for  improved  coordination. 

The  report  concludes  with  a  structured  and  annotated  list  of  34  current  re¬ 
search  issues  representing  the  core  questions  or  problems  currently  being 
discussed  or  researched  in  the  R&D  community. 
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Hemingway,  P.  W.,  and  Kubala,  A.  L.  A  comparison  of  speed  and  accuracy  of  1 

interpretation  of  two  tactical  symbologies.  Technical  Report  389 ,  j 

Ouly  1979.  (AD  A075  428)  1 


This  report  compares  the  speed  and  accuracy  of  interpretation  of  two 
tactical  symbologies.  One  set  of  symbols  was  the  standard  set  described  in 
U.S.  Army  Field  Manual  20-30.  The  other  set  was  designed  on  the  basis  of 
previous  research  in  symbology.  A  brief  battle  scenario  was  selected  and 
for  each  symbol  set  35mm  slides  were  prepared  representing  updates  in  a 
tactical  display.  Results  showed  that  (a)  accuracy  of  interpretation  did 
not  vary  between  the  sets,  (b)  the  author-designed  set  was  interpreted  more 
rapidly,  and  (c)  female  participants  were  more  rapid  in  interpreting  the 
displays  than  male  participants. 


Ciccione,  D.  S.,  Samet,  M.  G.,  and  Channon,  J.  B.  A  framework  for  the  de¬ 
velopment  of  improved  tactical  symbology.  Technical  Report  403,  August 
1979.  (AD  A076  017) 

The  utility  of  symbology  would  be  increased  by  making  it  more  compati¬ 
ble  with  current  and  emergent  tactical  doctrine.  A  structured  methodology 
based  on  a  role-playing  exercise  was  developed  for  eliciting  graphically  re¬ 
lated  tactical  information  requirements  from  experienced  military  tacticians. 

The  methodology,  involving  a  two-stage  elicitation  process,  employs 
doctrinally-sanctioned  information-processing  guidelines  to  prompt  partici¬ 
pants  to  generate  task-related  tactical  questions  (e.g.,  What  is  the  prin¬ 
cipal  deficiency  of  a  specific  enemy  unit?)  and  candidate  answers  (e.g., 
mobility,  personnel) .  A  pilot  test  used  two  experienced  staff  officers  and 
a  European  defensive  scenario.  The  numerous  question  and  answer  sets  ob¬ 
tained  were  organized  into  22  clusters  (i.e.,  data  structures) ,  with  each 
one  specifying  questions  in  decreasing  order  of  detail  according  to  a  common 
tactical  theme  (e.g.,  immediate  threat,  enemy  vulnerability,  priority  tar¬ 
gets)  .  These  data  structures  represent  categories  of  task-based  information 
requirements  which  can  serve  as  potential  building  blocks  in  the  development 
of  a  dynamic,  flexible  data  base  for  tactical  symbology. 
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The  framework  also  contains  a  preliminary  analysis  of  symbol  design 
effectiveness  based  on  a  taxonomy  of  basic  information-processing  behaviors, 
which  include  symbol  discrimination,  display  search,  and  symbol  learnability . 
Each  of  these  processing  components  was  used  as  a  focus  for  examining  rele¬ 
vant  research  literature  and  its  implications;  and  as  a  result,  preliminary 
guidelines  were  derived  for  improving  symbol  design  effectiveness  (e.g., 
minimize  the  amount  of  feature  similarity  among  different  mei  ’  ers  of  a  sym¬ 
bol  set) ♦  Finally,  to  help  monitor  future  symbology  development  efforts, 
a  multifaceted  evaluation  strategy  is  outlined  which  calls  for  systematic 
attention  to  content-based,  and  tactical  performance-based  assessment 
criteria. 


Farrell,  J.  P. ,  and  Halpin,  S.  M.  The  ability  to  extract  terrain  informa¬ 
tion  from  maps;  Factors  and  measures.  Technical  Report  494,  November 

1980.  '  "  ' 

This  paper  describes  two  experiments  used  to  test  the  validity  of  the 
Relief  Format  Assessment  Test,  a  battery  of  eight  tests  tapping  different 
aspects  of  map  understanding  for  evaluating  the  effectiveness  of  various 
alternate  map  formats  for  the  portrayal  of  terrain.  The  advantages  of  the 
Relief  Format  group  of  assessment  test  over  traditional  methods  of  evalua¬ 
tion,  survey,  and  field  testing  are  discussed. 

Experiment  I  is  a  Field  Validation  of  the  Relief  Format  Assessment  Test. 
Sixteen  military  personnel  (sergeant  to  captain)  were  tested  on  portrayal 
of  terrain  for  land  navigation  both  in  the  classroom  and  in  the  field.  Re¬ 
sults  from  Experiment  I  support  the  predictive  validity  of  the  eight  map 
tests  as  a  measurement  of  the  effectiveness  of  maps. 

Experiment  II  involved  Factor  Analysis  of  the  battery  of  tests  to  de¬ 
termine  the  psychological  abilities  involved  in  map  reading  performance. 
Fifty-nine  Army  personnel  were  administered  the  test3,  yielding  data  for 
factor  analyses.  Two  different  factor  analyses  were  conducted  each  yield¬ 
ing  two  factors.  One  factor  appears  to  involve  analytic  and  quantitative 
abilities,  while  the  second  involves  visual  factors. 

Implications  of  the  results  are  discussed. 


Landee,  B.  M. ,  Samet,  M.  G.,  and  Gellman,  L.  H.  User-oriented  tactical  in¬ 
formation  requirements  with  implications  for  graphic  portrayal  stan¬ 
dards.  Technical  Report  497 ,  November  1980. 

The  primary  purpose  of  this  research  was  to  elicit  and  organize  se¬ 
lected  battlefield  information  requirements  of  tactical  analysts,  and  to 
assess  the  adequacy  of  conventional  symbology  for  satisfying  these  require¬ 
ments.  An  elicitation  procedure  was  developed  and  utilized  with  small 
groups  of  Army  officers  in  a  tactical  role-playing  exercise  involving  a 
specified  Division-level  scenario.  Information  requirements  in  the  form 
of  tactical  questions  were  generated  and  reviewed  by  participants,  both 
individually  and  together.  In  addition,  for  each  question,  the  partici¬ 
pants  completed  a  form  which  provided  descriptive  information  such  as  whether 
the  answer  to  the  question  is  available  from  a  display  with  conventional 
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symbology  (FM  21-30).  In  an  effort  to  organize  the  information  requirements, 
an  hierarchical  clustering  technique  was  applied  that  analyzed  the  simi¬ 
larity  between  questions  in  terms  of  58  semantic  features  (e.g.,  friendly, 
enemy,  artillery,  vulnerability,  etc.). 

The  elicitation  sessions  resulted  in  the  generation  of  a  total  of  272 
disftinct  tactical  questions.  These  questions  formed  seven  major  clusters 
which  were  assigned  the  following  names  to  reflect  the  central  theme  of  their 
respective  information  content:  friendly,  enemy ,  time/ capability ,  status , 
activities/procedures ,  terrain/routes ,  and  planning.  The  descriptive  data 
about  the  questions,  analyzed  for  each  cluster  of  information  individually 
and  for  the  entire  set  of  questions,  indicated  that  conventional  symbology 
fails  in  many  respects  to  meet  basic  user  needs.  For  example,  43%  of  the 
information  requirements  identified  were  said  to  be  unavailable  from  a  con¬ 
ventional  display.  Furthermore,  there  appears  to  be  a  proliferation  of  per¬ 
sonalized  techniques  being  employed  by  users  to  portray  their  information 
requirements.  Some  major  informational  deficiencies  in  conventional  sym¬ 
bology  are  identified,  and  examples  are  given  on  how  the  findings  can  be 
employed  in  development  efforts  toward  making  standardized,  tactical  sym¬ 
bology  more  useful  and  effective. 

Samet,  M.  G. ,  Geiselman,  R.  E. ,  and  Landee,  B.  M.  An  experimental  evaluation 

of  tactical  symbol-design  features.  Technical  Report  498,  November  1980. 

Sixteen  nonmilitary  subjects  learned  each  of  two  symbol  sets  (conven¬ 
tional,  iconic)  to  criterion.  Each  set  contained  three  basic  symbols  repre¬ 
senting  unit  types  of  armor,  mechanized  infantry,  and  infantry,  respectively. 
After  learning  a  symbol  set,  each  subject  was  shown  a  series  of  situation 
displays,  where  some  displays  contained  symbols  coded  with  either  perimeter- 
density  or  vector  projection  to  convey  supplementary  unit-attribute  informa¬ 
tion  (unit  strength  or  firepower  reach).  For  each  display,  the  subject  was 
asked  questions  corresponding  to  different  behavioral  processes  (identifica¬ 
tion,  search,  comparison,  pattern  recognition) .  The  results  indicated  that: 
(a)  iconic  symbols  did  not  yield  faster  identification  performance  than  con¬ 
ventional  symbols,  and  conventional  symbols  yielded  faster  pattern-recognition 
performance  than  iconic  symbols;  (b)  the  portrayal  of  unit  attributes  slowed 
processing  of  unit-type  information  in  all  four  tasks,  but  vector  projection 
created  less  interference  than  perimeter  density  in  three  or  four  tasks; 

(c)  unit-strength  information  was  processed  faster  when  it  was  portrayed  as 
perimeter  density,  and  fire-power-reach  information  was  processed  faster 
when  portrayed  as  a  vector  projection.  On  a  further  task  which  required 
integration  of  several  symbols  into  an  analytical  judgment  (threat  value 
assessment) ,  performance  accuracy  was  found  to  be  insensitive  to  the  con¬ 
ventional  versus  iconic  symbology  comparison.  The  results  were  discussed 
in  terms  of  the  complexity  of  the  implications  involved  in  the  selection  of 
symbol  design  features  and  their  correspondence  with  tactical  concepts. 
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Topographic  Products  and  Procedures 

Granda,  T.  M.  A  comparison  between  a  standard  map  and  a  reduced  detail  map 
within  a  Simulated  Tactical  Operations  System  (SIMTOS) .  Technical 
Paper  274,  June  1976.  (AD  A028  752) 

The  abstract  for  this  reference  appears  on  page  III-22  of  the  section 
entitled  Graphic  Displays. 


Potash,  L.  M.  Design  of  maps  and  map-related  research.  Human  Factors,  1977, 

19,  139-150. 

Hardcopy  maps,  i.e.,  maps  printed  on  paper,  are  typically  multipurpose 
displays  that  have  to  meet  the  needs  of  a  variety  of  users.  They  must  por¬ 
tray  a  large  quantity  of  spatially  overlapping  geographic  information  which 
can  result  in  clutter  and  interference  among  symbols  if  the  map  is  not  well 
designed.  This  paper  will  summarize  and  organize  literature  related  to  the 
design  of  hardcopy  maps  and  map  research. 

The  first  portion  of  the  paper  defines  and  discusses  somo  characteris¬ 
tics  of  hardcopy  maps  which,  if  ignored,  can  lead  to  faulty  problem  defini¬ 
tion  and  to  research  having  little  applicability  to  hardcopy  maps.  Literature 
on  cartography,  visual  displays,  coding,  and  visual  perception  is  reviewed 
in  the  second  section  of  the  paper.  The  last  section  summarizes  research  on 
extant  maps. 


Potash,  L.  M.,  and  Jeffrey,  T.  E.  Factors  in  the  design  of  hardcopy  topo¬ 
graphic  maps.  Technical  Paper  284,  January  1978.  (AD  A049  621) 

A  literature  survey  on  factors  in  map  design,  including  visual  coding 
techniques  and  assessment  techniques,  is  summarized  in  this  report,  as  the 
first  step  in  improving  legibility  and  usefulness  of  hardcopy  (printed) 
topographic  maps. 

Map  design  determinants  are  scale,  interrelatedness  of  symbols,  and  a 
body  of  standardized  symbols  and  modes  of  coding.  Map  scale  influences 
fidelity,  as  small  scale  requires  selection,  simplification,  and  magnifica¬ 
tion  of  features.  Clutter  can  be  reduced  by  coding  to  differentiate  infor¬ 
mations  Color  coding  aids  in  identification  and  reduces  location  time? 
iconic  and  alphanumeric  shape  coding  can  be  learned  easily  and  are  flexible; 
size  coding  requires  considerable  space  and  increases  location  time. 

Among  map  assessment  techniques,  opinion  sampling  is  relatively  inex¬ 
pensive  but  does  not  measure  ac..ual  performance,  and  theoretical  analysis 
is  a  limited  first  step  only.  Empirical  analysis  measures  performance  with 
the  map  product,  either  by  assessing  performance  directly  or  by  measuring 
the  map-reading  skills  which  underlie  performance. 

In  comparisons  of  different  types  of  hardcopy  topographic  maps,  the 
best  photo-based  maps  produce  performance  comparable  to  that  with  conven¬ 
tional  line  maps.  Some  Army  users  preferred  to  augment  contour  lines  on 
conventional  maps  with  layer  tints  for  interpreting  topographic  relief. 
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Future  airborne  map  displays  should  consider  the  information  requirements 
of  pilots,  effects  of  vehicle  movement,  and  map  legibility  in  poor  light. 


Potash,  L.  M.,  Farrell,  J.  P.,  and  Jeffrey,  T.  E.  An  approach  to  assess¬ 
ment  of  relief  formats  for  hardcopy  topographic  maps.  Technical 
Paper  356,  April  1979.  (AD  A069  462) 

(Also:  A  technique  for  assessing  map  relief  legibility,  Cartographic 
Journal ,  1978,  15^  28-35) 

The  relief  assessment  test  described  in  this  report  is  designed  to  as¬ 
sess  the  effect  of  supplementing  contour  lines  with  other  relief  formats. 

It  contains  eight  types  of  problems  and  a  review  of  relevant  aspects  of  map 
reading.  It  has  been  produced  in  three  relief  formats:  contour  lines, 
contour  lines  plus  layer  tints,  and  contour  lines  plus  shading.  Initial 
work  indicates  that  addition  of  layer  tints  to  contour  lines  can  increase 
speed  of  reading  some  types  of  relief  information?  whereas,  addition  of 
shaded  relief  does  not  increase  map  reading  speed  more  than  use  of  layer 
tints  and  can  cause  a  decrease  in  accuracy. 

Map  designers  will  be  interested  in  this  report. 


Landee,  B.  M.,  Samet,  M.  G.,  and  Foley,  D.  R.  A  task-based  analysis  of 

information  requirements  of  tactical  maps.  Technical  Report  397, 

August  1979.  (AD  A086  502) 

A  task-based  approach  for  specifying  and  analyzing  map  information  re¬ 
quirements  was  developed  and  demonstrated.  A  review  of  selected  literature 
concerning  the  mapping  process  included  methods  for  selecting  environmental 
data  to  be  shown  on  a  map.  Seven  tactical  tasks,  representing  common  battle¬ 
field  functions  performed  by  different  users  and  echelons  (e.g.,  determina¬ 
tion  of  enemy  avenues  of  approach  by  a  G2  officer  at  the  division  level) , 
were  sampled  and  analyzed  in  depth.  A  military  role-playing  and  doctrinal 
verification  procedure  was  used  to  divide  tasks  into  subtasks  so  that  cor¬ 
responding  information  requirements  could  be  specified.  Each  subtask 
(e.g.,  identification  of  obstacles  to  movement),  in  turn,  was  broken  down 
into  basic  tactical  questions  about  the  environment  (e.g.,  "Are  there  any 
vegetated  areas  through  which  the  enemy  cannot  pass?, "  "Are  there  are  slopes 
enemy  vehicles  cannot  climb?") .  The  data  required  to  answer  each  question 
defined  the  information  categories  (e.g.,  vegetated  areas,  slopes)  and  levels 
of  details  (e.g.,  tree  spacing  and  trunk  diameter,  percent  slope)  necessary 
for  successful  task  completion. 

The  resulting  map-related  information  requirements  were  synthesized  to 
generate  representative  map  development  guidelines.  This  synthesis— which 
emphasized  information  about  vegetation,  road  networks,  and  built-up  areas — 
was  accomplished  across  tasks  to  identify  those  specific  information  needs 
which  show  either  prominent  commonality  or  uniqueness  with  respect  to  dif¬ 
ferent  tasks  and  user  groups.  Within  the  framework  of  these  task -based 
comparisons,  examples  illustrate  the  types  of  implications  that  can  be  de¬ 
rived  from  the  task-based  analysis  of  information  requirements. 
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Farrell,  J.  P. ,  and  Potash,  L.  M.  A  comparison  of  alternate  formats  for 
the  portrayal  of  terrain  relief  on  military  maps.  Technical  Report 
42B,  November  1979.  (AD  A081  739) 

Two  experiments  evaluated  the  effectiveness  of  relief  shading  and  layer 
tints  as  supplements  to  contour  lines  for  solving  eight  types  of  military 
map-use  problems.  The  addition  of  layer  tints  was  found  to  increase  the 
speed  of  solving  most  problems  with  no  loss  in  accuracy.  The  addition  of 
shaded  relief  increased  the  time  required  to  solve  problems  requiring  de¬ 
tailed  information  but  may  be  valuable  in  providing  a  quick  general  im¬ 
pression.  Personnel  experienced  in  using  military  maps  solved  map  problems 
more  accurately  but  no  faster,  on  the  whole.  Also,  solving  military  map 
use  problems  in  the  classroom  appears  to  correlate  with  performance  in  the 
field. 
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Surveillance  and  Reconnaissance  Management 

Bedarf,  E.  W.  Responses  to  job  assignment  and  preparation/training  ques¬ 
tions  by  G2  Air  and  image  interpretation  personnel.  Research  Memoran¬ 
dum  72-5,  October  1972. 

Summaries  are  presented  of  responses  to  sections  of  two  questionnaires 
administered  to  G2  Air  and  image  interpreter  personnel.  Data  were  taken 
from  the  Aerial  Surveillance  and  Reconnaissance  Questionnaire  for  G2  per¬ 
sonnel  and  a  questionnaire  of  the  same  name  administered  to  image  inter¬ 
preters.  The  survey  was  conducted  as  part  of  an  effort  to  determine  the 
job  requirements  of  the  two  categories  of  personnel,  documented  in  Research 
Report  1181. 


Youngling,  E.  W. ,  Vecchiotti,  R.  A.,  Bedarf,  E.  W. ,  and  Root,  R.  T.  Job 
requirements  of  G2  Air  and  image  interpretation  personnel.  Research 
Report  1181,  May  1974.  (AD  780  815) 

The  purpose  of  this*  effort  was  to  determine  the  tasks,  duties,  and  as¬ 
sociated  procedures  of  G2  Air  and  image  interpretation  personnel.  In  prac¬ 
tice,  the  effort  served  to  identify  (a)  problems  in  the  current  Army  air  in¬ 
telligence  system,  and  (b)  requirements  of  new  systems  that  necessitate 
training  modifications.  Data  were  collected  using  three  methods,  namely 
(a)  a  comprehensive  literature  review,  (b)  in-depth  interviews  with  knowledge¬ 
able  Army  personnel,  and  (c)  mailed  questionnaires  sent  to  various  Army  aerial 
surveillance  and  reconnaissance  units  around  the  world.  The  data  were  inte¬ 
grated  within  the  framework  of  the  Army  air  intelligence  system  by  using  sys¬ 
tem  flow  diagrams  and  task  analyses. 

The  main  findings  of  the  study  focus  on  the  G2  Air  officer  and  the  G2 
Air  officer  position.  The  G2  Air  officer  is  mainly  a  manager  of  resources, 
but  the  personnel  who  fill  the  positions  receive  inadequate  training  to  pre¬ 
pare  them  for  the  gob.  There  is  a  need  to  fill  the  G2  Air  officer  position 
only  with  appropriately  trained  personnel.  A  need  also  exists  for  tactical 
commanders  to  gain  a  better  understanding  of  the  capabilities  and  limita¬ 
tions  of  the  aerial  reconnaissance  and  surveillance  system  and  the  role  of 
the  G2  Air  personnel. 

Secondary  findings  suggest  a  need  for  more  balance  in  training  image 
interpretation  personnel.  There  are  tasks  for  which  more  than  an  adequate 
amount  of  training  is  given  in  comparison  to  the  importance  of  the  task. 

Other  problems  with  the  aerial  reconnaissance  and  surveillance  system  in¬ 
volving  equipment  and  procedures  have  also  been  identified. 


Bedarf,  E.  W.,  and  Potash,  L.  M.  A  field  evaluation  of  the  Aerial  Surveil¬ 
lance  and  Reconnaissance  MANAGER.  Research  Memorandum  75-14,  December 
1975.  (AD  AO 7 6  792)  ’ 

A  handbook,  Aerial  Surveillance  and  Reconnaissance  MANAGER,  had  been 
developed  from  an  earlier  analysis  of  the  responsibilities  and  duties  of 
Army  G2  Air  officers,  reported  in  Research  Report  1181.  The  handbook  was 
given  field  evaluation,  using  a  survey  technique.  Questionnaires  were  sent 
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to  practicing  G2  Air  officers,  training  instructors  in  pertinent  courses, 
and  individuals  in  closely  related  positions  who  had  previously  received 
copies  of  the  handbook. 

Almost  all  those  responding  perceived  the  primary  function  of  the  hand¬ 
book  as  that  of  reference  book  and/or  as  a  training  aid.  The  major  disad¬ 
vantage  cited,  that  not  enough  information  was  given,  may  have  resulted 
partially  from  the  fact  that  the  manual  was  designed  to  serve  both  purposes. 


Vecchiotti,  R.  A.,  Berrey,  J.  L. ,  and  Bedarf,  E.  W.  Development  of  resource 
management  materials  for  the  G2  Air  officer.  Technical  Paper  333, 
September  1978.  (AD  A061  695) 

A  major  conclusion  of  a  study  of  the  job  requirements  for  G2  Air  and 
image  interpretation  personnel  (ARI  Research  Report  1181)  stressed  the  need 
to  direct  more  attention  to  the  training  of  the  G2  Air  officer.  This  present 
effort  concentrated  on  developing  resource  management  materials  for  the  G2 
Air  officer  that  could  be  used  for  on-the-job  training  and  guidance  in  the 
performance  of  duties  and  in  intelligence  school  courses. 

Resource  management  materials  were  compiled  in  the  form  of  a  handbook 
to  achieve  the  training  objectives.  The  first  step  involved  gathering  in¬ 
formation  on  tasks  as  performed  by  operational  aerial  surveillance  and  re¬ 
connaissance  units,  preparing  a  comprehensive  study  data  base,  and  conduct¬ 
ing  field  observations  and  interviews  to  verify  the  data  base.  Next,  a 
content  outline  of  the  handbook  was  prepared  by  integrating  the  field  in¬ 
terview  data  with  the  existing  data  base.  Knowledgeable  personnel  reviewed 
this  outline.  Third,  preparation  of  the  handbook  itself  took  into  account 
the  various  training  techniques  and  aids  appropriate  for  on-the-job  and 
school  application,  step  four  consisted  of  conducting  a  limited  evaluation 
of  the  handbook  to  determine  its  usefulness,  acceptance,  and  final  structure. 
Next,  a  "preliminary  edition"  of  the  handbook  used  the  information  derived 
from  the  evaluation.  Finally,  an  automated  demonstration  of  a  portion  of 
the  handbook  illustrated  how  the  materials  could  be  used  with  possible  future 
computerized  information  systems. 


Vecchiotti,  R.  A.,  Berrey,  J.  L. ,  and  Narva,  M.  A.  Training  in  utilization 

of  surveillance  and  reconnaissance  resources  l>v  combat  arms  officers. 

Technical  Paper  325,  September  1978.  (AD  Obi  577) 

The  purpose  of  this  effort  was  to  increase  the  effective  U3e  of  aerial 
surveillance  and  reconnaissance  (AS(iR)  resources  by  combat  commanders  through 
analysis  of  current  experience  and  training  and  subsequent  development  of 
materials  that  may  be  used  for  training  and  field  use. 

Contemporary  training  materials  were  analyzed  for  content. 

Questionnaires  pertaining  to  experience  with,  attitudes  toward,  and 
training  received  relative  to  AS&R  were  distributed  to  students  at  pertinent 
schools,  and  the  responses  were  analyzed  for  implications  for  training. 
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Combat  arms  students  receive  limited  training  in  the  use  of  AS&R  re¬ 
sources,  and  they  felt  they  could  use  more  information  in  this  area*  The 
more  experience  with  AS&R,  the  more  favorable  has  been  the  experience  with 
this  resource.  More  emphasis  should  be  placed  on  practical  exercises  that 
permit  appropriate  interaction  with  the  capabilities  of  AS&R. 

The  results  of  the  analyses  were  used  in  the  subsequent  development  of 
training  materials  and  a  field  aid. 


Shvern,  U.  Field  evaluation  of  the  combat  commander’s  guide  to  aerial  sur¬ 
veillance  and  reconnaissance  resources.  Technical  Paper  380,  July 

1979.  (AD  A075  422) 

This  research  evaluated  the  Combat  Commander's  Guide  to  Aerial  Surveil¬ 
lance  and  Reconnaissance  Resources  on  its  usefulness,  accuracy,  clarity, 
and  completeness. 

An  evaluation  questionnaire  was  prepared  and  distributed,  together  with 
copies  of  the  Commander's  Guide,  to  100  officers  at  U.S.  Amy  units  in  Korea, 
Germany,  Fort  Hood,  Tex.,  and  Fort  Bragg,  N.C.  Participants  were  required 
to  have  either  command  experience  at  the  battalion  level  or  above  or  G3/S3 
experience.  The  questionnaire  included  a  structured  evaluation  of  the  main 
sections  of  the  Commander's  Guide  as  well  as  a  global  assessment  of  the 
document.  Participants  could  also  elaborate  further  on  their  answers  or 
comment  on  topics  not  specifically  addressed  in  the  questionnaire.  Sixty 
questionnaires  were  returned — 27  from  battalion  commanders  and  executive 
officers  and  33  primarily  from  officers  in  G3/S3  assignments. 

The  individual  sections  of  the  Commander's  Guide  were  judged  to  be  more 
useful  than  the  document  as  a  whole.  This  seemingly  paradoxical  result  was 
obtained  with  both  the  structured  rating  scales  and  the  open-ended  comments. 
Raters  considered  the  Commander's  Guide  to  be  more  suitable  for  the  G2/S2 
than  for  the  battalion  or  brigade  commander.  Raters'  comments  concerning 
possibly  excessive  detail  for  a  commander's  needs  are  consistent  with  this 
finding.  Reference  to  specific  AS&R  assets  -\re  outdated  in  many  instances. 
Rapid  changes  in  the  AS&R  area  dictate  frequent  revision  of  any  publication 
that  refers  to  specific  equipment. 

The  anticipated  frequency  of  use  of  the  Commander's  Guide  would  increase 
greatly  during  wartime.  If  the  Commander's  Guide  is  revised,  fundamental  re¬ 
structuring  would  be  required  to  achieve  high  use  rates  by  commanders  during 
peacetime  conditions.  A  consideration  of  the  type  of  AS&R  information  that 
commanders  are  able  and  willing  to  use ,  as  opposed  to  what  they  should  ideally 
use,  would  be  required,  otherwise  a  technically  excellent  publication  might 
remain  unused  on  the  commander's  shelf  or  be  passed  on  to  his  G2  or  S2.  Un¬ 
less  the  user  has  become  familiar  with  the  Commander's  Guide,  its  effective¬ 
ness  in  an  emergency  is  limited. 
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King,  R.  B.,  Rochford,  D.  s.,  Ray,  T.  E.,  and  Bedarf,  E.  W.  Development  of 
a  test  bed  for  aerial  surveillance  and  reconnaissance  resource  manage- 
ment.  Working  Paper  HF-80-4,  November  1979. 

Recent  job  requirements  analysis  of  the  duties  of  Army  G2  Air  officers 
indicates  that  more  attention  should  be  given  to  training  of  management 
techniques.  In  response  to  this  need,  a  handbook  (Aerial  Surveillance  and 
Reconnaissance  MANAGER)  was  developed  to  serve  as  an  on-the-job  reference 
and  training  aid.  9 

This  project  developed  a  test  bed  for  the  evaluation  of  the  MANAGER 
handbook  and  other  resource  management  materials  utilized  by  G2  Air  officers. 

The  test  bed  consists  of  a  basic  scenario  interspersed  with  91  multiple  ^ 

choice  performance  measurement  items.  Emphasis  is  given  to  management  and 
decision-making  processes  and  test  items  are  divided  among  eight  G2  Air  task 
functions. 

The  test  bed  exists  in  both  hardcopy  and  automated  form.  The  automated 
version  is  designed  for  computer  administration  at  the  AR1  Training  and  In¬ 
formation  Systems  Facility. 


Shvern,  U. ,  and  Coe,  A.  Collection  resources  allocation  planning  systems 
Draft  user's  manual  fox  desk-top  computer  (IBM  5100)  application. 
Working  Paper  HF-80-6,  February  1980. 

This  paper  describes  procedures  for  utilization  of  a  prototype  collec¬ 
tion  planning  system. 


Special  Reports 

Aerial  Surveillance  and  Reconnaissance  MANAGER,  1973. 

This  document  integrates  information  useful  to  the  management  duties 
of  G2  Air  officers.  Emphasis  has  been  placed  on  depicting  the  flexibility 
that  often  exists  in  the  management  of  Aerial  Surveillance  and  Reconnaissance 
(AS&R)  resources.  The  duties  and  the  AS&R  resources  discussed  are  those  at 
the  division  and  corps  levels. 


Combat  commander's  guide  to  Aerial  Surveillance  and  Reconnaissance  resources, 

1974. 

This  document  describes  the  capability  of  the  Aerial  Surveillance  and 
Reconnaissance  (AS&R)  system.  It  was  intended  primarily  for  combat  command¬ 
ers  at  battalion  and  brigade  levels  who  need  an  overall  understanding  of  how 
to  utilize  the  system.  Closely  related  "field  aids"  were  also  prepared  for 
each  of  three  combat  arms  areas — Armor,  Artillery,  and  Infantry. 

Commander's  field  aid  to  Aerial  Surveillance  and  Reconnaissance 
utilization,  Armor,  1974. 
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Coronander's  field  aid  to  Aerial  Surveillance  and  Reconnais sai.ee  utili 
sat ion,  Artillery,  1974. 

Consnander's  field  aid  to  Aerial  Surveillance  and  Reconnaissance  utili 
zation,  Infantry,  1974. 
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Remotely  Monitored  Sensors 

Edwards,  L. ,  Rochford,  D.  S.,  and  shvern,  U.  Comparison  of  four  unattended 
ground  sensor  displays.  Technical  Paper  281,  April  1977.  (AD  A039  056) 

The  research  presented  here  compared  the  relative  effectiveness  of  four 
different  unattended  ground  sensor  (UGS)  displays--an  X-T  plotter,  and  three 
variations  of  the  situation  map  display — in  terms  of  their  effect  on  monitor 
performance. 

Four  2-hour  UGS  scenarios  were  compiled  from  previously  recorded  field 
tests  of  military  targets.  The  recordings  were  played  back  to  activate  the 
displays  during  experimentation.  Sixteen  Naval  personnel  (eight  relatively 
experienced  with  UGS  and  eight  inexperienced)  were  given  training  on  the 
displays.  Each  subject  then  monitored  each  display  in  turn  for  2  hours, 
reporting  target  information  as  he  would  operationally.  The  reports  were 
compared  to  known  ground  truth  and  were  scored  on  total  detections,  false 
alarms,  detection  accuracy,  and  target  direction  (i.e.,  direction  of  target 
movement) . 

Operator  performance  was  unaffected  by  the  type  of  display  used.  Op¬ 
erators  were  able  to  detect  a  higher  percentage  of  targets  during  periods 
of  low  target  activity  them  during  periods  of  high-target  activity.  How¬ 
ever,  the  accuracy  of  the  detections  was  higher  during  the  high  activity 
period.  Levels  of  experience  did  not  have  a  significant  effect  on 
performance. 

Edwards,  L. ,  Pilette,  S.,  and  Martinek,  H.  Error  analysis  of  unattended 

ground  sensor  operators'  reports.  Research  Memorandum  77-1,  November 
1977.  (AD  A077  921) 

This  analysis  was  done  £o  locate  the  chief  causes  of  the  relatively 
frequent  errors  made  by  operators  monitoring  grid  arrays  of  unattended 
ground  sensors  (UGS) .  Error  analyses  were  made  of  UGS  operator  performance 
data  from  three  previous  studies.  A  table  categorized  the  errors  and  showed 
dependent  and  independent  variables  considered  ss  error-causing  factors. 
Difficulty  indexes  and  target  profiles  of  the  thiee  projects  were  calculated 
to  pinpoint  error-causing  factors.  Observations  and  calculated  trends  iden¬ 
tified  highly  probable  error-causing  situations. 

The  results  showed  greater  errors  occurring  (a)  during  increased  tar¬ 
get  activity  and  target  load,  (b)  because  of  false  alarms  during  low  target 
activity  and  low  target  load,  (c)  because  of  the  effects  of  single  and  mul¬ 
tiple  vehicle  target  patterns  on  speed  and  direction  estimation,  (d)  in  de¬ 
termining  direct  and  indirect  target  trails,  (e)  because  of  operacor  inat¬ 
tention  associated  with  sequence  of  presentation  of  the  target,  (f)  during 
variations  in  certain  X-T  plot  pattern  features,  and  (g)  because  of  combined 
errors  in  measurement,  calculation,  and  response  requirement  management. 
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Martinekj  H.,  Pilette,  S.,  and  Biggs,  B.  The  effects  of  work/rest,  tar¬ 
get  activity,  background  noise  and  string  size  on  operator  interpre- 
>tat ion  of  unattended  ground  sensor  records.  Technical  Paper  300, 

June  1978..  (AD  A061  043) 

This  research ...  (a)  investigated  the  relationships  between  unattended 
ground  (Sensor*  (UGS)  operator  performance  and  work/rest  cycles,  sensor 
string  size,  target  activity  levels,  and  noisy/quiet  environments  and 
(b)  identified  sources  of  operator  error  that  can  be  eliminated  through 
the  use  of  new  interpretation  techniques ,  procedures,  and  training. 

Operators  monitored  UGS  records  for  8  hours  of  worktime  under  four 
different  work/rest  conditions :  2  hours  work,  1  hour  rest;  2  hours  work, 

15  minutes  rest;  4  hours  work,  1  hour  rest;  and  4  hours  work,  15  minutes 
rest.  The  UGS  records  included  counterbalanced  vaxictLiuus  of  three  string 
sizes  (2,  3,  and  4  sensors),  12  targets  per  hour  or  6  targets  per  hour, 
and  high  and  low  "battle"  noise. 

For  a  contemplated  8  hours  of  monitoring,  2-hour  work  shifts  inter¬ 
spersed  with  1-hour  or  15-minute  rest  periods  are  to  be  preferred  over 
4-hour  work  shifts  interspersed  with  1-hour  or  15-minute  rest  periods . 

For  heavy  short-term  monitoring  requirements ,  for  example ,  enemy  attack 
requiring  periods  of  concentrated  monitoring,  an  operator  can  perform 
satisfactorily  for  one  4-hour  shift  but  performance  will  deteriorate  dur¬ 
ing  a  second  4-hour  shift. 

Performance  was  equal  using  2,  3,  or  4  sensor  springs  in  the  low-target 
activity  condition.  However,  during  the  high-target-activity  condition,  use 
of  3  or  4  sensor  strings  resulted  in  more  target  detections  as  compared  to 
use  of  2  sensor  strings. 

Operator  performance  during  the  high  "battlefield"  noise  was  equal  to 
that  during  low  noise.  During  the  high-target-activity  conditions  the 
operators  detected  more  targets  than  during  the  low-target-activity  condi¬ 
tion.  However,  they  detected  a  higher  percentage  of  targets  during  the  low 
condition. 


Martinek,  H.,  Hilligoss,  R.  E.,  and  Lavicka,  F.  Comparison  of  three  dis¬ 
play  devices  for  unattended  ground  sensors.  Technical  Paper  299, 
August  1978.  (AD  A060  558) 

This  report  describes  an  experiment  designed  to  determine  the  relative 
values  of  three  devices  for  displaying  activations  of  seismic  unattended 
ground  sensors  (UGS) :  the  operational  RO  376  event  recorder,  a  situation 
map  display,  and  a  time-compressed  situation  map  display. 

Three  tape  recordings,  each  2  hours  long,  of  the  UGS  activations  were 
compiled  from  recordings  taken  during  field  tests.  Twelve  Naval  personnel 
experienced  in  the  use  of  the  RO  376  were  given  4  hours*  training  in  the 
use  of  the  situation  map  display  and  in  the  map  display  used  with  time  com¬ 
pression.  Each  participant  then  interpreted  each  display  using  a  different 
set  of  recorded  activations  each  time,  in  counterbalanced  order,  and  filled 
out  a  standard  report  form. 
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Use  of  the  RO  376  resulted  in  higher  accuracy  and  more  Complete  re¬ 
ports  than  did  use  of  the  other  displays.  No  differences  were  found  be¬ 
tween  the  two  modes  of  situation  map  display. 


Pilette,  S.,  Biggs,  B.,  and  Martinek,  H.  The  value  of  special  training  and 

job  aids  for  improving  unattended  ground  sensor  operator  performance. 

Technical  Paper  304,  August  1978.  (AD  A061  046) 

This  research  was  conducted  to  determine  the  value  of  unattended  ground 
sensor  operator  training  and  job  aids  derived  from  an  analysis  of  errors 
and  to  determine  the  effect  on  operator  performance  of  target-activity  level 
and  number  of  sensors  used  in  a  string . 

The  training  program  and  job  aids  were  evaluated  using  the  pretest  and 
posttest  design  with  4  hours  of  scenarios  containing  typical  target  patterns 
at  two  levels  of  target  activity  generated  by  2-,  3-,  and  4-sensor  strings. 
Operators  detected  and  identified  targets  and  estimated  their  speed  and 
number. 

The  individualized  training  program  resulted  in  significantly  improved 
operator  interpretation  performance  in  target  detection  rights,  identifica¬ 
tion  rights,  target  speed ,  and  target  quantity  estimation.  Use  of  the  job 
aid  (nomograph)  significantly  improved  performance  in  identification  rights, 
target  speed  estimation,  target  quantity  estimation,  and  reporting  time. 
Student  acceptance  of  the  individualized  training  approach  and  both  job 
aids  was  high.  .Operator  performance  pn  3-  and  4-sensor  strings  was  77% 
detection  completeness  with  virtually  no  false  alarms.  Detection  of  tar¬ 
gets  was  better  during  low-target  .activity  than  during  high-target  activity. 
The  use  of  three  sensors  in  a  string  resulted  in  the  same  operator  perfor¬ 
mance  as  the  use  of  four  sensors. 


Edwards,  L, ,  Pilette,  S.f  Biggs,  U . ,  and  Martinek,  H.  The  effect  of  work¬ 
load  on  performance  of  operators  monitoring  unattended  ground  sensors. 

Technical  Paper  321,  September  1578.  (AD  AQ61  694) 

The  purpose  of  the  research  was  to  determine  the  effect  of  workload 
(target  activity  level  and  number  of  sensors  used)  on  unattended  ground 
sensor  (UGS)  operators'  target-detection  ability,  false-alarm  rate,. and  di¬ 
rection  and  speed  estimation  accuracy. 

Experienced  operators  monitored  activations  of  UGS  used  in  grids  under 
six  workload  conditions--two  leveli  of  target  activity  x  three  levels  of 
number  of  sensors  monitored.  Target  activity  level  was  defined  as  either 
5  or  14  vehicle  groups  per  27  sensors  per  hour.  " The  operators  monitored 
in  counterbalanced  sequence  27  sensors  (3  grids),  54  sensors  (6.  grids) , 
and  108  sensors  (12  grids). 

Percentage  of  detections  decreased  linearly  from  85%  (4  to  6  vehicle 
groups  per  hour  presented  on  27  sensors)  to  26%  (about  40  vehicle  groups 
per  hour  presented  on  108  sensors) .  Without  special  training,  operators 
should  monitor  no  more  than  60  sensors  being  activated  by  a  maximum  of  10 
vehicular  groups  per  hour  (detection  completeness  for  this  condition  was 
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76%).  The  number  of  false  alarms  and  the  accuracy  of  speed  estimation  did 
not  significantly  change  over  workload.  Operator  accuracy  in  direction 
estimation  decreased  significantly  as  a  function  of  workload  from  the  azi¬ 
muth  +300  to  true  azimuth  ±54°. 

Martinek,  H.,  Pilette,  S.,  and  Biggs,  B.  Vehicle  identification  using  the 
acoustic  sensors  Training,  sensing  concepts  and  bandwidth.  Technical 
Paper  334,  September  1978.  (AD  A062  601) 

Experiments  were  designed  to  (a)  develop  and  validate  a  training  pro¬ 
gram  for  using  the  acoustic  sensor  to  identify  vehicles  in  convoy,  (b)  esti¬ 
mate  operator  performance  in  identifying  vehicles  using  the  acoustic  sensor, 
and  (c)  investigate  the  effect  of  using  different  sensing  concepts  and  band- 
widths  on  the  operator’s  ability  to  identify  vehicles. 

V 

Following  orientation  and  procedure  training ,  18  school-trained  unat¬ 
tended  ground  sensor  (UGS)  operators  were  tested  bn  their  ability  to  identify 
vehicles  in  convoys.  Their  training  used  immediate  feedback,  self-scoring, 
paired  comparisons,  and  practice.  Following  this  training,  operators  were 
retested. 

The  training  package  increased  operator  vehicle  identification  perfor¬ 
mance  by  46%  to  16%,  depending  on  the  level  of  target  detail  required. 


Pilette,  S.,  Biggs,  B.,  Edwards,  L.,  and  Martinek,  H.  Optimum  patching 
technique  for  seismic  sensors  employed  in  a  grid.  Technical  Paper 
320,  September  1978.  (AD  A061  573) 

The  requirement  was  to  determine  the  effect  of  four  patching  techniques 
and  associated  job  aids,  training,  sensor  density,  and  target  activity  on 
operator  performance  in  detecting  targets  when  using  unattended  ground  sen¬ 
sors  employed  in  a  grid  configuration. 

Trained  operators  interpreted  tactical  data  recorder  (RO  376)  plots 
both  before  and  after  training  using  four  patching  techniques  and  associated 
job  aids  in  the  grid  employment  of  seismic  sensors.  The  plots  represented 
both  9  sensors  per  square  kilometer  grid  and  24  sensors  per  square  kilometer 
grid.  Completeness  of  target  detection,  number  of  false  alarms,  speed  esti¬ 
mation  error,  and  target  direction  deviation  were  measured. 

Patching  technique  training  (and  job  aids)  increased  detection  com¬ 
pleteness  by  42% s  The  row  patching  technique  was  selected  as  the  best.  Use 
of  the  9-sensor  grid  resulted  in  fewer  false  alarms  and  equal  detection  com¬ 
pleteness  as  compared  to  use  of  the  24-sensor  grid. 


Pilette,  S.,  Biggs,  B.,  and  Martinek,  K.  The  value  of  special  training  for 
the  interpretation  of  unattended  ground  sensors  employed  in  a  grid. 
Technical  Paper  328,  October  1978.  (AD  A063  594) 

Based  on  an  error  analysis  of  unattended  ground  sensor  operator  target 
detection  data,  a  self-paced  training  program  was  developed  to  reduce  the 
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frequency  and. magnitude  of  operator  errors  ’  and  increase,  target  detection 
rate.  The  first  training  unit  dealt  with  solitary  targets— only  one  vehicle 
or  vehicle  convoy  traveling  through  the  grid*.  The  second  dealt  with  target 
clusters — several  vehicles  or  Vehicle  convoys  traveling  through  the  grid 
at  one  time..  '  . " 

To  assess  the  value  of  training  a  pre-posttest  design  was  used.  Op¬ 
erators  monitored  for  2  hours  activation  data. from  seismic  .sensors  employed 
in  a  grid,  received  the  special  trailing, , and  again :  monitored V-fpr  2  hours. 
The  two  monitoring  periods  were  counterbalanced  to  control  for  differences 
in  the  target  activation  data.  Four  workload  levels  (numbers  of  targets 
activating  the  sensors)  were  used  to  ascertain  training  effects  across, a 
range  of  workloads  typical  of  operational  conditions.  ■  *  '  <}' 

The  self-paced  training  program  resulted  iri  improved  operato^.perfor-.; 
mance  in  target  detection  and  target  speed  and  direction  estimation  accuracy 
of  both  practical  and  statistical  significance .  Target  detection  improved 
from  58%  to  79%,  averaged  over  all  conditions.  Target  detection  after, train 
ing  averaged  61%,  66%,  94%,  and  97%  for  the  four  workload  condition?  (27, 

15,  8,  and  5  targets  Aour) .  Speed  and  direction  estimation  improved  by  23% 
and  20%  respectively  as  a  result  of  training.  The  false  alarm  rate  was 
negligible  under  all  conditions.  Conclusions  ar.d  recommendations  for  the 
implementation  of  this  training  program  and  for  . Utilization  of  grid  deploy¬ 
ment  are  provided . 


Huntoon,  R.  B. ,  Schohan,  B. ,  and  Shvern,  U.,  Visual  search  performance  in 
simulated  remotely  piloted  vehicle  utilization  as  a  function  of  aux¬ 
iliary  task  loading  on  the  observer.  Technical  Paper  357,  April  1979, 
(AD  A072  402) 

Baseline  data  were  obtained  on  how  well  observers  could  extract  infor¬ 
mation  from  a  TV  monitor  while  performing  auxiliary  tasks  under  task  load¬ 
ing  conditions  that  might  be  encountered  in  use  of  a  remotely  piloted  ve¬ 
hicle  (RPV)  as  the  sensor  platform., 

A  simulation  facility  was  used.  It  contained  a  terrain  model,  a  TV 
camera  transport,  hybrid  computing  equipment,  and  a  television  display  and 
control  console. 

Six  pilots  and  six  nonpilots  participated  in  the  three-phase  effort. 
Phase  A  required  participants  to  detect  and  recognize  tank-sized  targets  in 
open  and  cluttered. backgrounds  from  a  simulated  altitude  of  2,000  feet  and 
a  simulated  RPV  velocity  of  100  knots.  Phase  B  required  participants  to 
monitor  and  correct  deviations  in  the  RPV  course  and  altitude  and  to  respond 
to  two  visual  warning  indicators.  Phase  B  tasks  were  presented  at  two  rates 
one  per  10  saconds  and  three  per  10  seconds.  Phase  C  combined  the  tasks  of 
Phase  A  and  Phase  B  with  concurrent  task  demands  upon  the  participants. 

Increasing  the  auxiliary  load  level  decreased  the  probabilities  and 
ranges  of  target  detection  and  recognition.  Target  acquisition  task  de¬ 
mands  similarly  increased  auxiliary  task  response  times.  Cluttered  back¬ 
ground  significantly  degraded  target  acquisition  task  performance,  particu¬ 
larly  when  the  auxiliary  task  was  performed  concurrently. 
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Pilette,  S.,  Biggs,  B.,  and  Mart inek,  H.  Requirements  for  target  identifi¬ 
cation  training  for  the  acoustic  sensor  operator.  Research  Memorandum 
79-4,  April  1979.  (AD  A077  977) 

The  unattended  ground  sensor  (UGS)  operator  in  the  field  has  few  oppor¬ 
tunities;  for  formal  ^  or  structured  practice  in  identifying  vehicles 

usinfcf  the  acoustic  sensor.  Actual  operator  performance  levels  are  unknown. 
To  assist  in  defining  requirements  for  operator  training,  this  research  was 
designed  to  estimate  the  current  performance  level  of  operators  under  the 
ideal  conditions  of  identifying  vehicles  traveling  alone  (not  in  convoys) 
using  an  acoustic  sensor  system  with,  a  high  signal/noise  ratio.  In  addi¬ 
tion,  the  time  required  to  identify  a  vehicle  and  the  effects  of  additional 
training  were  investigated. 

Twenty- four  UGS  operators  identified  120  vehicle  sounds  which  varied 
by  type  of  vehicle  (8)  and  length  of  time  presented  (1,  3,  6,  10,  and  15 
seconds),  received  2  hours  of  training,  and  then  wero  retested  on  the  origi- 
•  nal  120  sounds . 

The  training  produced  large  increases  in  operator  performance  (average 
of  31%),  and  decreased  the  time  required  to  identify  a  vehicle  (from  15  sec¬ 
onds  to  6) .  Practice  effects  were  found,  indicating  that  practice  is  needed 
in>  the  field  and  that  training  research  in  this  area  must  be  designed  so 
that  training  effects  are  not  confounded  with  practice.  Operators  required 
6  to  10  seconds  to  identify  vehicles  after  training,  depending  on  the  spe¬ 
cific  vehicle  type.  Selection,  using  readily  obtained  measures,  of  the  top 
third  of  UGS  operators  resulted  in  a  25%  increase  in  performance  over  that 
of  the  "average  operator." 
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Martinek,  H. ,  Pilette,  S.,  and  Biggs,  B.  The  effects  of  signal/noise  ratio 

and  bandwidth  on  the  vehicle  identification,  using  the  acoustic  sensor. 

Technical  Paper  377,  June  1979.  (AD  A073  715) 

Three  experiments  were  conducted  to  determine  the  effect  of  variations 
in  signal-to-noise  (S/N)  ratio  and  increased  bandwidth  on  the  ability  of 
remotely  monitored  sensor  operators  using  the  acoustic  sensor  to  identify 
vehicles  in  convoy.  In  general,  the  operator  was  to  discriminate  between 
the  following  seven  military  vehicles  traveling  in  typical  convoys:  jeeps, 
gamma  goats,  2‘5-ton  trucks,  5-ton  trucks,  10-ton  trucks,  armored  personnel 
carriers,  and  tanks.  Targets  were  presented  at  each  of  four  levels  of  S/N 
ratio:  +6  decibels  (dB) ,  +12  dB,  +18  dB,  and  +24  dB.  The  operational 
bandwidth  of  50-2,000  hertz  (Hz)  was  compared  to  that  of  50-4,500  Hz. 

Special  training  under  all  of  the  above  conditions  was  given. 

The  results  indicate  that  operator  identification  completeness  declines 
as  the  S/N  ratio  decreases  approximately  1%  per  1.5  dB  of  S/N  ratio.  No 
differences  were  found  in  use  of  the  two  bandwidths.  Use  of  automatic  gain 
control  should  be  limited  because  the  operator  uses  loudness  variations  to 
discriminate  among  targets. 
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Martinek,  H.  Summary  of  ARI  research  on  remotely  monitored  sensors.  Re¬ 
search  Report  1.253,  July  1980. 

One  relatively  new  source  of  intelligence  information  is  remote  moni¬ 
toring  of  the  battlefield,  using  seismic,  acoustic,  and  magnetic  remotely 
monitored  sensors  (REMS) .  When  enemy  personnel  or  vehicle  movement  acti¬ 
vates  these  remote  sensors,  a  monitor  display  located  behind  our  lines 
indicates  the  activity.  The  operator  can  derive  from  this  display  not  only 
the  enemy's  presence  but  also  such  information  as  direction  and  speed  of 
convoys  and  personnel,  number  of  vehicles  in  a  convoy,  and  convoy  composi¬ 
tion — e.g.,  armored  versus  wheeled  vehicles. 

This  publication  summarizes  ARI  research  on  user  problems  of  REMS,  in¬ 
cluding  direct  operational  applications  of  present  and  future  utilization  of 
REMS.  Major  findings  are  categorized  into  five  areas — training,  operator 
aids,  operational  procedures,  REMS  system  design,  and  personnel  requirements. 
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UGS  nomograph  and  ruler,  1974. 

This  device  was  developed  for  the  purpose  of  quick  and  accurate  calcu¬ 
lation  of  vehicular  ground  speed  based  on  information  obtained  from  unat¬ 
tended  ground  sensors. 
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Operator's  training  workbook:  Readout  problem  areas  for  UGS  string  employ¬ 
ment  (with  instructor's  guide),  1975. 

Based  on  an  analysis  of  operator  errors,  a  training  program  was  devel¬ 
oped  that  stresses  individualized  instruction  including  self-pacing,  immedi¬ 
ate  feedback,  expert  assistance  when  needed,  and  guaranteed  student-mastery 
using  criterion  testing.  The  eight  content  areas  are:  the  UGS  ruler, 
2-sensor  strings  and  formula  review,  final-points,  column-length  quantity, 
irregular  activations  patterns,  EMID,  MAGID,  and  DIRID. 

The  individualized  training  program  resulted  in  significantly  improved 
operator  interpretation  performance  in  target  detection  rights,  identifica¬ 
tion  rights,  target  speed,  and  target  quantity  estimation.  Student  accep¬ 
tance  of  the  individualized  training  approach  was  nigh.. 


Operator's  training  workbook:  Readout  problem  areas  for  UGS  grid  employ¬ 
ment  (with  instructor's  guide),  1976. 

Based  on  an  error  analysis  of  unattended  ground  sensor  operator  target 
detection  data,  a  self-paced  training  program  was  developed  to  reduce  the 
frequency  and  magnitude  of  operator  errors  and  increase  target  detection 
rate.  The  first  training  unit  dealt  with  solitary  targets — only  one  vehicle 
or  vehicle  convoy  traveling  through  the  grid.  The  second  dealt  with  target 
clusters — several  vehicles  or  vehicle  convoys  traveling  through  the  grid  at 
one  time. 

The  self-paced  training  program  resulted  in  improved  operator  perfor¬ 
mance  in  target  detection  and  target  speed  and  direction  estimation  accuracy 
of  both  practical  and  statistical  significance.  Target  detection  improved 
from  58%  to  79%,  averaged  over  all  conditions.  Target  detection  after  train¬ 
ing  averaged  61%,  66%,  94%,  and  97%  for  the  four  workload  conditions  (27, 

15,  8,  and  5  targets/hour) .  speed  and  direction  estimation  improved  by  23% 
and  20%  respectively  as  a  result  of  training.  The  false  alarm  rate  was 
negligible  under  all  conditions.  Conclusions  and  recommendations  for  the 
implementation  of  this  training  program  and  for  utilization  of  grid  deploy¬ 
ment  are  provided. 


Target  identification  training  using  the  acoustic  remote  sensor  (with  train¬ 
ing  tape  recording),  1976. 

The  five-part  training  program  is  recorded  on  tape  and  can  be  used  as 
a  self-  or  a  group-administrable  package.  Part  I  involves  a  short  instruc¬ 
tional  briefing,  followed  by  a  two-convoy  exercise  in  which  the  operators 
identified  the  vehicles.  Feedback  is  provided  to  the  operators  by  giving 
them  the  correct  vehicle  identification.  The  operators  record  the  vehicle 
names  on  their  target  logs  so  that  they  can  score  their  interpretations. 

The  operators  then  are  given  the  same  convoys  to  analyze  again  as  they  ob¬ 
serve  their  target  logs  with  the  correct  identifications  marked  on  them. 

Part  II  involves  a  comparison  of  the. sound  of  one  vehicle  with  that 
of  another  immediately  following.  For  each  paired  comparison,  the  vehicle 
identities  are  given  before  the  sounds.  Each  of  the  target  types  (vehicles) 
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is  compared  with  every  other  tatget  type  for  a  total  of  21  paired  compari¬ 
sons.  All  target  vehicles  are  traveling  in  the  fast  condition— about  40 
kilometers  per  hour  (kmph)  or  24  miles  per  hour  (mph) .  This  exercise  is 
designed  to  help  the  operators  remember  how  the  vehicles  sound  in  relation 
to  one  another. 

Part  III  involves  the  same  procedures  as  in  Part  I,  except  that  two 
convoys  are  used.  This  exercise  gives  the  operators  practice  on  the  same 
vehicle  types  but  with  different  individual  vehicles  and  with  different 
combinations  and  variations  in  signal/noise  ratio  and  loudness. 

Part  IV  involves  the  same  matched-pairs  procedure  as  Part  II.  However, 
the  target  vehicles  are  traveling  slower  than  in  Part  II,  about  20  kmph  or 
12  mph.  This  exercise  is  designed  to  give  operators  a  chance  to  compare  the 
sound  of  one  slow-moving  vehicle  with  another,  a  distinction  required  because 
the  sound  signatures  of  vehicles  can  differ,  depending  upon  speed. 

Part  V  involves  a  replay  of  the  four  convoys  that  the  operators  had 
previously  worked  with.  The  convoys  are  administered  in  random  sequence, 
and  the  operators  again  report  vehicle  types  on  the  target  logs.  Feedback 
is  then  given,  and  the  operators  score  themselves.  The  training  was  shown 
to  significantly  improve  operator  performance. 
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Image  Interpretation 

Birnbaum,  A.  H.,  Sadacca,  R.  ,  Andrews,  R.  S.,  and  Narva,  M.  A.  Summary  of 

BESRL  surveillance  research.  Technical  Research  Report  1160,  September 

1969.  (AD  701  907) 

The  Surveillance  Systems  research  program  of  the  U.S.  Army  Behavioral 
Science  Research  Laboratory  (BESRL — now  ARI)  had  as  its  objective  the  pro¬ 
duction  of  scientific  data  bearing  on  the  extraction  of  information  from 
surveillance  displays  and  the  efficient  storage,  retrieval,  and  transmis¬ 
sion  of  this  information  within  an  advanced  computerized  image  interpretation 
facility.  The  present  technical  research  report  summarizes  in  integrated 
fashion  the  major  problem  areas,  the  rationale  of  BESRL' s  approach  to  their 
solution,  and  the  general  course  of  research  studies  completed  or  in  progress 
in  the  surveillance  areas  of  manned  systems  experimentation.  The  research 
effort  is  conducted  within  the  following  fields:  (a)  Interpreter  Techniques — 
the  determination  of  interpreter  techniques  in  a  surveillance  facility; 

(b)  Image  Interpretation  Displays — influence  of  displays  on  image  interpreter 
performance;  (c)  Intelligence  Systems — intelligence  information  processing 
systems;  (d)  Image  Systems — information  processing  in  advanced  image  inter¬ 
pretation  systems.  Surveillance  Systems  research  programs  have  resulted  in 
findings  which  are  applicable  in  optimizing  human  component  performance  in 
existing  systems  and  in  providing  systems  developers  with  information  useful 
in  design  specifications  for  future  systems. 


Samet,  M.  G.  Checker  confidence  statements  as  affected  by  performance  of 

initial  image  interpreter.  Technical  Research  Note  214,  September 

1969.  (AD  700  127) 

Experiments  on  interpreter/checker  performance  have  generally  concen¬ 
trated  on  completeness,  accuracy,  and  timeliness  as  measures  of  individual 
and  team  proficiency.  With  the  advent  of  computer-aided  intelligence  sys¬ 
tems,  emphasis  on  techniques  for  processing  probabilistic  information  has 
grown.  The  present  publication  reports  on  one  aspect  of  assigning  inter¬ 
preters  to  work  as  two-man  teams  in  which  one  interpreter  checks  interpre¬ 
tations  made  independently  by  his  teammate.  The  study  was  specifically 
concerned  with  determining  how  different  levels  of  identification  accuracy 
and  of  confidence  validity  associated  with  an  initial  interpreter  affect 
the  confidence  validity  of  the  checker.  Four  equivalent  imagery  sets  of  60 
annotated  targets  were  used  in  the  experimental  procedure.  The  first  set 
of  60  targets  was  assigned  to  18  newly  trained  interpreters  in  an  initial 
performance  test  which  required,  the  subjects  to  supply  identifications  and 
confidence  statements  for  each  annotation.  Target  identifications  and  con¬ 
fidence  statements  attributed  to  an  initial  checker  were  developed  for  each 
of  the  three  remaining  sets  of  imagery.  In  the  preliminary  test  phase, 
confidence  estimates  were  stated  under  a  point  payoff  scheme  in  which  it  was 
to  the  disadvantage  of  the  interpreter  to  overstate  or  understate  his  con¬ 
fidence.  Half  of  the  interpreters  were  given  feedback  on  individual  per¬ 
formance.  In  the  experiment  proper,  each  interpreter  was  given  three  sets 
of  preanno  ated  imagery  with  associated  hypothetical  confidence  statements 
which  he  was  required  to  examine,  note  previous  identification/confidence 
information,  and  then  state  his  own  confidence  in  the  given  identification. 
Task  performance  was  accomplished  under  the  point  payoff  condition. 
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Analysis  of  data  obtained  on  seven  derived  measures  yielded  the  fol¬ 
lowing  findings:  (a)  Checkers  typically  improved  on  the  confidence  validity 
of  interpreters  who  were  poor  or  only  moderately  good  in  stating  confidence, 
(b)  Checker  confidence  statements  in  identifications  made  by  interpreters 
with  "excellent"  record  of  confidence  validity  were  less  valid  than  those  of 
initial  interpreters,  (c)  Interpreter's  confidence  statements  were  more 
valid  when  they  were  checking  than  when  they  were  stating  confidence  in 
their  own  identifications.  (d)  Checker's  confidence  statements  were  more 
affected  by  observed  variations  in  identification  accuracy  of  the  initial 
interpreter  than  by  his  confidence  validity.  (e)  Receipt  of  feedback  on 
initial  identification  and  confidence  proficiency  had  no  effect  on  subse¬ 
quent  checker  performance. 


Levine,  J.  M. ,  Feallock,  J.  B.,  Sadacca,  R. ,  and  Andrews,  R.  S.  Method  for 
quantifying  subjective  costs  of  large  numbers  of  image  interpretation 
errors.  Technical  Research  Note  218,  November  1969.  (AD  704  706) 

Reliable  quantitative  estimates  of  the  cost  of  erroneous  items  of  sur¬ 
veillance  information  are  needed  so  that  intelligence  requirements  of  field 
commanders  can  be  stated  with  greater  precision.  This  research  developed  a 
practical  and  rapid  method  for  measuring,  in  the  field,  the  subjective  costs 
that  unit  commanders  attach  to  the  consequences  of  decisions  based  upon  dif¬ 
ferent  errors  or  deficiencies  in  image  interpretation.  Nearly  3,500  inter¬ 
pretation  errors  were  defined  by  means  of  a  hierarchical  target  classifica¬ 
tion  and  reporting  scheme  constructed  for  the  study.  Magnitude  Estimation 
(ME)  was  chosen  as  the  feasible  method  for  determining  the  interval  scale 
for  costs  for  such  a  large  number  of  errors.  Costs  were  measured  for  a 
sample  of  several  hundred  errors  directly,  based  on  combat  officer  judgments 
and  costs  of  the  remaining  errors  were  predicted  by  multiple  regression 
analysis.  Cost  estimates  obtained  by  applying  two  scaling  procedures  and 
the  ME  procedure  were  compared  statistically  to  evaluate  scale  validity, 
reliability,  and  sensitivity.  Direct  cost  estimations  yielded  mean  validity 
coefficients  of  .67  and  .73  based  on  the  two  criterion  scales,-  mean  relia¬ 
bility  was  .80.  The  ME  scale  was  sensitive  to  differences  in  the  tactical 
situation — offensive  versus  defensive — for  which  the  aerial  surveillance 
information  was  required.  The  utility  of  error  cost  matrices  in  military 
situations  is  discussed. 


Cockrell,  J.  T.  Maintaining  target  detection  proficiency  througli  team  con¬ 
sensus  feedback.  Technical  Research  Note  219,  December  1969. 

(AD  707  376) 

An  exploratory  study,  reported  on  in  Technical  Research  Note  195, 
"Maintaining  Image  Interpreter  Proficiency  Through  Team  Consensus  Feedback," 
was  designed  to  assess  the  usefulness  of  the  team  consensus  feedback  process 
as  a  possible  aid  to  proficiency  maintenance  for  interpreters  in  an  image 
interpretation  facility.  The  objective  of  the  present  experiment  was  to 
determine  if  the  target  detection  skill  of  individual  interpreters  can  be 
improved  by  feedback  which  team  members  generate  for  themselves  as  they 
compare  and  discuss  their  work.  This  experiment  differed  from  the  explora¬ 
tory  study  in  that  only  target  detection  was  required,  rather  than  detection 
plus  identification.  In  addition,  the  experimenter  investigated  the  impact 
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on  individual  interpreter  performance  of  (a)  size  of  team  (three-man  versus 
two-man) >  (b)  discussion  versus  no  discussion;  (c)  initial  proficiency 
level  of  team  members;  and  (d)  team  composition  (heterogeneous  versus  homo¬ 
geneous)  with  respect  to  initial  proficiency  level.  Sixty  USAIS  graduates 
participated  in  the  experiment.  Treatment  was  a  3-day  practice  session. 

A  pre-  and  posttreatment  test  was  administered  to  each  interpreter  to  as¬ 
sess  detection  proficiency.  Interpreters  assigned  to  feedback  conditions 
practiced  in  teams  and  were  permitted  to  either  discuss  or  compare  their 
work;  the  no-feedback  interpreters  practiced  alone  and  were  not  permitted 
to  discuss  or  compare  their  work  with  anyone.  Neither  group  received  ground 
truth  feedback  under  the  experimental  procedure. 

It  was  found,  as  in  previous  experiments,  that  interpreters  working  in 
teams  with  consensus  feedback  showed  greater  improvement  than  interpreters 
working  alone  in  reducing  inventive  errors;  there  was  no  difference,  however, 
in  errors  of  omission.  No  difference  obtained  between  discussion  versus  no¬ 
discussion  and  three-man  teams  versus  two-man  teams,  but  interpreters  work¬ 
ing  in  heterogeneous  teams  showed  significantly  greater  gain  in  performance 
on  all  measures  than  interpreters  on  homogeneous  teams.  Findings  also  indi¬ 
cated  a  relatively  greater  improvement  in  performance  of  team  members  who 
are  initially  low  in  proficiency  than  those  who  are  initially  high  in  pro¬ 
ficiency.  From  both  experiments  conducted  to  date,  evidence  points  to  the 
effectiveness  of  team  consensus  feedback  in  maintaining  and  enhancing  per¬ 
formance  of  interpreters  in  field  units,  particularly  in  target  identifica¬ 
tion  and  reduction  of  inventive  errors.  The  technique  appears  to  be  es¬ 
pecially  useful  where  ground  truth  is  not  available. 


Jeffrey,  T.  E.  Unaided  reading  of  coded  reconnaissance  data.  Technical 

Research  Note  220,  December  1969.  (AD  707  377) 

The  general  objective  of  the  research  was  to  determine  the  feasibility 
of  having  image  interpreters  decode  sensor  and  terrain  information  encoded 
on  imagery  by  direct  viewing  instead  of  resorting  to  use  of  elaborate  code 
reading  machines.  The  investigation  sought  specifically  to  determine: 

(a)  speed  with  which  image  interpreters  can  decode  designated  positions 
of  the  code  matrix  block  (CMB)  data;  (b)  accuracy  of  decoding  performance; 

(c)  usefulness  of  CMB  magnification  as  a  means  of  enhancing  the  interpreter's 
speed  and  accuracy  in  decoding  matrix  content.  Subjects  in  the  experiment 
(16  graduate  image  interpreters  plus  one  experienced  interpreter)  read  por¬ 
tions  of  15  different  code  matrix  blocks  arranged  in  3  sets  of  5  blocks  each 
on  which  reconnaissance  information  was  encoded.  The  interpreters  had  been 
trained  to  recognize  spatial  patterns  of  dots  represent  !.ng  the  information 
to  the  point  of  2  error-free  performances.  Three  5-mat'  groups  decoded  each 
matrix  block,  each  set  under  1  of  3  different  levels  o':  magnification — 
single  power  (unaided  eye),  2-power  (stereoscope),  and  .-power  (tube 
magnifier) . 

In  the  experiment  proper,  achievement  was  measured  in  terms  of  time  re¬ 
quired  for  interpreters  to  locate  the  required  block,  decode  and  record  the 
data,  and  number  of  correct  decodings.  Analysis  of  the  data  obtained  indi¬ 
cated:  (a)  Average  interpreter  was  98%  accurate  in  translation  from  code  to 

clear  language;  (b)  direct  inspection  (unaided  eye)  was  not  significantly 
aided  by  magnification;  (c)  use  of  seven-power  magnification  reduced  decoding 
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time  over  two-power  magnification;  (d)  practice  significantly  reduced  de¬ 
coding  time;  accuracy  was  not  affected.  Findings  suggest  utility  of  estab¬ 
lishing  training  procedures  and  providing  fla&h  cards  for  interpreter 
practice  and  improvement  in  recognizing  the  spatial  patterns  used  in  encod¬ 
ing  reconnaissance  data.  In  addition,  equipping  the  seven-power  magnifica¬ 
tion  now  issued  in  the  PI  Kit  with  a  reticle  designed  to  aid  in  defining 
tha  data  fields  would  be  useful. 


Epstein.  S.  Effects  of  image  incongruence  on  location  of  common  terrain 

in  comparative  cover.  Technical  Research  Note  222,  February  1970. 

(AD  707  441) 

The  study  was  designed  to  determine  the  effects  of  differences  in  photo 
scale  and  orientation  of  early  and  late  cover  and  of  variation  in  percent 
of  overlap  on  inventive  error,  completeness,  and  speed  with  which  image  in¬ 
terpreters  identify  the  comparative  cover  photographs  and  confidence  in  re¬ 
ports.  In  the  experiment  96  trained  image  interpreters  were  given  the  task 
of  locating,  on  rolls  of  new  photographic  cover  (late  cover) ,  frames  depict¬ 
ing  the  same  terrain  as  that  presented  in  the  early  cover.  There  were  12 
experimental  conditions  involving  2  levels  of  difference  in  photo  scale 
(1:1  and  1:2),  3  levels  of  variation  in  orientation  (identical  or  0°,  30°, 
and  180°)  and  2  levels  of  percent  of  overlap  (100%  and  50%) .  A  Greco-Latin 
square  counterbalanced  arrangement  was  employed  in  presenting  the  subjects 
(24  groups  of  4  interpreters  each)  with  the  search  rolls  and  the  early  cover 
frame.  Interpreter  performance  was  measured  in  terms  of  the  following  cri¬ 
teria:  (a)  time — number  of  seconds  to  locate  the  relevant  late  cover  frames, 

(b)  frame  location  completeness — percent  of  relevant  search  roll  frames 
correctly  selected,  (c)  inventive  errors — number  of  frames  incorrectly  se¬ 
lected,  and  (d)  confidence — expression  of  self-confidence  in  responses  rated 
on  a  3-step  scale. 

The  inventive  error,  completeness,  speed,  and  confidence  of  interpreter 
performance  seem  to  be  seriously  affected  when  comparative  cover  varies  in 
scale  and  orientation  and  is  rot  completely  overlapping.  Specific  effects 
were:  Frame  location  completeness  decreased  with  variation  in  amount  of 
overlap  and  with  increased  scale  and  orientation  discrepancy,  overlap  dif¬ 
ferences  having  the  greatest  effect.  Search  time  increased  significantly 
with  reduced  overlap  but  was  less  affected  by  discrepancy  in  orientation 
and  photo  scale.  Inventive  errors  increased  with  reduced  overlap  and  with 
greater  scale  and  orientation  variation,  overlap  difference  again  having 
the  greatest  effect.  Although  reduced  overlap  and  orientation  discrepancy 
decreased  interpreter  confidence  significantly,  scale  discrepancy  had  no 
appreciable  effect.  Findings  suggest  that,  in  operational  practice,  degrad¬ 
ing  effects  of  image  incongruence  upon  interpreter  performance  in  identifying 
comparative  cover  can  be  minimized  by  the  availability  of  variable  magnifi¬ 
cation  and  image  rotation  capabilities  in  the  display  system. 


Martinek,  H.,  and  Bigelow,  G.  F.  Compendium  of  BESRL  performance  measures 
for  image  interpretation  research.  Research  Study  70-1,  April  1970. 

The  BESRL  (now  ARI)  Technical  Support  Branch  film  library  maintained 
a  large  number  of  image  interpreter  performance  measures  based  on  imagery 
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in  various  media  acquired  and  analyzed  by  branch  personnel.  The  present 
publication  provides  a  listing  of  image  interpretation  performance  measures 
for  which  roll  film  was  available  in  the  library  as  of  1  January  1970. 

For  each  performance  measure,  identification  and  description  were  pro¬ 
vided  under  the  following  rubrics: 

Identification.  Roll  numbers?  number  of  rolls  and  number  of  frames 
in  each  roll?  consecutive  exposures  (bursts)  or  individual  scenes. 

Use_.  Appropriate  use  and  reference  to  relevant  BESRL  research 
publications. 

Content .  Exercise  on  which  flown?  terrain?  target  areas?  target 
types  included. 

Technical  Data.  Format,  scale,  quality,  ground  resolution,  annota¬ 
tions,  stereo/nonstereo,  transparencies,  positive/negative,  how  reproduced? 
flight  information. 

Availability.  Form  in  which  available  for  reproduction  and  use. 

The  imagery  and  the  film  rolls  developed  for  performance  measures  have 
potential  utility  for  future  studies  in  image  interpretation  by  BESRL  re¬ 
searchers  and  research  scientists  in  other  organizations.  This  compendium 
is  designed  to  provide  prospective  users  with  sufficient  information  for 
them  to  identify  measures  which  are  suitable  for  specific  research 
objectives. 


Cockrell,  J.  T. ,  and  Sadacca,  R.  Training  individual  image  interpreters 
using  team  consensus  feedback.  Technical  Research  Report  1171,  June 
1971.  (AD  747  R27)  ‘  ~  "  ”  ' 

Four  experiments  explored  various  team  consensus  feedback  methods  with 
varying  conditions  of  image  interpreter  team  size  and  composition.  The 
experiments  were  directed  to  developing  team  practice  methods  which  would 
lead  to  the  greatest  performance  gains  by  individual  interpreters  in  an 
advanced  interpretation  system. 

The  first  experiment  was  designed  to  obtain  a  general  assessment  of 
the  usefulness  of  the  consensus  feedback  process  in  both  target  detection 
and  target  identification.  The  second  experiment  sought  to  determine  if 
team  consensus  feedback  would  be  effective  in  enhancing  target  detection 
oerfortnance  only.  Experiment  III  evaluated  modified  team  consensus  feed¬ 
back  techniques  designed  to  minimize  feedback  delays.  Experiment  IV  was 
concerned  mainly  with  the  nature  of  learning  curves  using  team  consensus 
feedback  techniques  and  with  evaluation  of  the  most  effective  procedures 
identified  in  the  first  three  experiments. 

Team  consensus  feedback  was  demonstrated  to  be  an  effective  method 
of  maintaining  and  improving  image  interpreter  proficiency  in  an  opera¬ 
tional  image  interpretation  facility.  Greatest  performance  gains  were  made 
in  target  identification.  Results  showed  that  interpreters  who  were 
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initially  low  in  proficiency  achieved  the  most  significant  gains  in  per¬ 
formance.  Low  proficiency  interpreters  learned  from,  or  benefited  through 
their  collaboration  with,  higher  proficiency  interpreters.  Neither  team 
discussion  nor  team  size  significantly  affected  interpreter  performance; 
however,  these  variables  did  combine , to  produce  ar^  overall  effect.  The 
ideal  procedure  seems  to  be  one  using  thrae-man  .'t^ams ,  heterogeneous  in 
terms  of  initial  proficiency,  in  which  individuals  first  perform  initial 
interpretation  on  an  individual  basis  and  then  discuss  their  identifications 
freely  in  deciding  on  the  team  report. 

Jeffrey,  T.  E.  Vertical  photographic , coverage  obtainable  with  varying  film 
format,  film  footage,  lens  focal  length,  altitude,  overlap,  and  side- 
lap.  Research  Memorandum  77-2,  June  1972.  (Ab  A079  384) 

Two  tables  are  presented  that  make  it  possible  to  obtain  an  estimate  of 
the  linear  and  area  coverage  obtainable  on  25  feet  of  70  mm  film  using  a 
vertical  camera  equipped  with  one  of  three  specified  focal  length  lenses, 
at  altitudes  ranging  from  500  feet  to  21,000  feet,  with  or  without  overlap 
on  successive  exposures  and  with  or  without  sidelap  between  successive  runs 
of  the'  mission.  Table  1  is  used  to  determine  the  denominator  of  the  photo 
scale  of  the  imagery.  This  value  is  called  the  Photo  Scale  Reciprocal  (PSR) 
PSR  is  used  to  enter  Table  2  to  obtain  the  desired  area  coverage.  Area 
coverage  is  given  tr  square  meters  and  for  four  different  conditions — no 
overlap  and  no  sidelap,  no  overlap  and  10%  sidelap,  60%  overlap  and  no  side- 
lap,  and  60%  overlap  and  10%  sidelap.  The  effect  of  film  format  on  the 
amount  of  ground  coverage  is  not  directly  given,  but  can  be  estimated  from 
Table  2  data.  Table  1  can  be  used  to  determine  the  mission  altitude  and/or 
lens/focal  length  that  must  be  used  to  obtain  imagery  of  the  desired  scale. 

Narva,  M.  A.  Consideration  of  the  AR-85A  viewer-computer  for  use  in 

computer-aided  instruction  in  image  interpretation.  Research  Memoran¬ 
dum  72-3,  June  1972,  (AD  A079  385) 

The  feasibility  of  using  the  AR-85A  Viewer-Computer  as  a  vehicle  for 
administering  computer-assisted  training  in  image  interpretation  is  assessed 
The  Viewer-Computer,  designed  to  be  used  for  mensuration  and  reporting  func¬ 
tions,  includes  a  limited  computer  capacity  with  associated  limited  input- 
output  capabilities.  When  this  capability  is  not  being  used  for  its  princi¬ 
pal  functions,  it  would  possibly  be  used  as  a  tool  for  instruction.  The 
most  promising  use  would  appear  to  be  in  identification  training  and  in  the 
presentation  of  feedback  to  the  trainee.  Small  units  of  instruction  might 
be  feasibly  incorporated  at  various  points  in  the  program  of  the  individual 
trainee. 


Lepkowski,  J.  R.,  and  Jeffrey,  T.  E.  Some  factors  affecting  mensuration 
variability  among  image  interpreters.  Research  Memorandum  72-7, 
September  1972.  (.'0  A079  387) 

From  results  of  this  exploratory  study,  the  following  tentative  con¬ 
clusions  were  drawn:  (a)  The  use  of  interpreter  scales  or  reticles  gradu¬ 
ated  to  thousandths  of  a  foot  or  in  millimeters  had  no  significant  effect 
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on  mensuration  variability,  (b)  Measurement  variability  for  targets  of  the 
size  used  in  these  experiments  did  not  vary  significantly  with  target  ground 
size,  (c)  Imagery  scale  had  no  significant  influence  on  the  variability  of 
target  measurement.  The  significant  interaction  obtained  in  the  four-factor 
analysis  between  target  size  and  imagery  scale  was  probably  an  artifact. 

(d)  Magnification  level  had  no  significant  effect  on  mensuration  varia¬ 
bility.  (e)  Measurements  made  using  reticles  appeared  to  be  less  variable 
than  those  made  with  an  interpreter  scale,  (f)  Interpreters  tended  to  main¬ 
tain  their  relative  position  from  measurement  task  to  measurement  task  with 
respect  to  the  mean  measurement  of  the  group. 


Martinek,  H.,  Hilligoss,  R.  E. ,  and  'lerrington,  B.  Effectiveness  of  an 
error  key  for  image  interpretation  in  Vietnam.  Technical  Research 
Note  230,  September  1972.  (AD  752  437) 

Experimentation  to  develop  a  specialized  reference  key  to  enable  inter¬ 
preters  to  avoid  certain  common  errors  in  reporting  information  derived  from 
imagery  taken  over  Vietnam  is  reported  here.  The  research  was  undertaken  to 
construct  and  validate  a  new  type  of  reference— an  error  key — for  use  in 
Vietnam.  The  key  was  constructed  from  operational  imagery  of  Vietnam  terri¬ 
tory  with  annotations  pointing  out  types  of  error  to  be  avoided.  Evaluation 
was  conducted  using  trained  interpreters  to  determine  if  use  of  the  error 
key  would  achieve  the  following  effects  on  image  interpreter  reports: 

(a)  increase  number  of  right  responses;  (b)  decrease  number  of  wrong  re¬ 
sponses;  and  (c)  increase  accuracy  of  identification. 

Performance  results  achieved  by  122  enlisted  image  interpreters  (recent 
II  Course  graduates)  were  used  in  evaluating  effectiveness  of  the  error  key. 
Each  of  two  matched  groups  interpreted  two  sets  of  imagery,  one  set  using 
the  error  key,  the  other  without  using  the  key.  Analysis  of  results  showed 
that  (a)  the  BESRL  error  key  significantly  improved  the  accuracy  of  inex¬ 
perienced  interpreters  in  dealing  with  operational  imagery  on  Vietnam — 39% 
increase  in  rights,  26%  decrease  in  errors  made,  and  43%  increase  in  accuracy 
rate;  (b)  the  error  key  significantly  improved  the  completeness  of  reporting 
on  the  subject  imagery;  and  (c)  differences  in  performance  as  a  function  of 
mission  and  group  were  evident. 

For  optimal  operational  use ,  the  present  key  should  be  extended  -to  in¬ 
clude  different  areas  of  Vietnam,  different  scales  and  types  of  photography, 
and  different  sensors  such  as  infrared.  There  is  apparent  advantage  to  be 
obtained  in  use  of  the  key  in  the  field  (Vietnam)  and  for  training  in  the 
Image  Interpretation  (II)  Course. 


Martinek,  H.,  and  Hilligoss,  R.  E.  Accuracy  and  completeness  of  interpreta¬ 
tion  as  a  function  of  time  for  selected  conditions.  Research  Memoran¬ 
dum  72-6,  October  1972. 

Assignment  of  image  interpreter  personnel  and  procedures  applied  must 
ultimately  take  into  account  the  time  required  for  varying  degrees  of  accu¬ 
racy  and  completeness  of  interpretation.  A  beginning  was  made  in  an  effort 
to  determine  the  relationship  of  accuracy,  completeness,  and  time  spent  in 
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interpretation  under  a  variety  of  conditions  and  to  obtain  some  parametric 
data  concerning  these  variables.  Curves  are  presented  showing  these 
relationships . 


Jeffrey,  T.  E.,  and  Beck,  F.  J.  Intelligence  information  from  total  opti¬ 
cal  color  imagery.  Research  Memorandum  72-4,  November  1972. 


This  report  describes  an  evaluation  of  total  optical  color  (TOC)  imagery 
as  a  source  of  intelligence  information.  The  imagery  consisted  of  aerial 
surveillance  photographs  acquired  over  the  UNDERBRUSH  test  range  at  Eglin 
Air  Force  Base.  The  Total  Optical  Color  (TOC)  35  mm  system  was  compared  to 
a  system  providing  conventional  color  imagery  and  to  a  system  providing 
monochromatic  imagery.  The  only  performance  index  showing  significant  dif¬ 
ference  between  chromatic  and  achromatic  color  presentation  was  time  taken 
to  extract  information  from  a  slide.  Black-and-white  imagery  required 
about  20%  longer  than  the  color. 


Information  extraction  can  be  accomplished  more  rapidly  from  chromatic 
than  from  achromatic  color  imagery.  The  time  savings  is  accomplished  with 
no  decrease  in  the  accuracy  or  completeness  of  target  detection  and  identi¬ 
fication.  Subjects  expressed  a  preference  for  color  imagery  over  black-and- 
white,  and  for  imagery  on  conventional  color  film  over  the  achromatic  color 
imagery  from  the  TOC  system. 


Narva,  M.  A.  Evaluation  of  selected  pictorial  characteristics  of  reference 
materials  for  use  in  image  interpretation.  Technical  Research  Note  233, 
November  1972.  (AD  754  567)  \ 

Three  related  experiments  investigated  the  characteristics  of  pictorial 
content  of  reference  materials  (keys)  used  by  image  interpreters  with  a  view 
to  determining  the  most  effective  way  of  representing  objects  in  the  key. 

The  set  of  experiments  was  concerned  with  obtaining  information  pertaining 
to  the  optimal  manner  of  presenting  recognition  features  in  a  key  so  as  to 
aid  an  interpreter  in  final  identification  of  an  object  seen  in  imagery. 

Each  experiment  involved  different  combinations  of  the  characteristics  under 
study — (a)  type  of  presentation  (use  of  photographs  or  outline  drawings,  or 
both) ,  (b)  viewing  angle  (vertical,  oblique,  or  both) ,  and  (c)  scale  of  the 
image  in  the  key  (large  or  small) .  In  the  first  experiment,  computer-aided 
procedures  for  selecting  the  category  of  the  object  imaged  were  included. 

In  experiments  two  and  three,  no  computer  aids  were  employed.  The  inter¬ 
preter  used  only  the  key  which  contained  no  textual  material.  In  each  ex¬ 
periment,  recently  graduated  image  interpreters  identified  a  series  of  16 
vehicles  organized  into  four  sets  and  presented  in  a  balanced  research  de¬ 
sign.  Two  levels  of  quality  were  used  in  the  test  imagery. 

Performance  was  more  rapid  with  photographs  than  with  line  drawings 
when  the  key  was  used  with  a  computer-assisted  category  selection  procedure. 
When  the  key  was  used  alone,  no  difference  between  photographs  and  drawings 
was  found  in  speed  or  in  number  of  correct  identifications.  No  advantage 
was  obtained  in  presenting  more  than  one  viewing  angle  nor  by  presenting 
photographs  and  schematic  representations  together.  Reduced  scale  in  the 
key  images  required  greater  identification  time,  but  did  not  result  in 
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greater  accuracy.  A  net  result  of  the  experimentation  is  to  permit  greater 
leeway  in  the  materials  included  in  keys  and  in  the  manner  of  presentation. 


Epstein,  S.,  and  Jeffrey,  T.  E.  Common  area  demarcation,  target  annotation, 

and  target  lists  as  aids  in  change  detection.  Technical  Research  Note 

238,  March  1973.  (AD  761  128) 

E'.perimentation  to  improve  performance  in  detecting  changes  that  may 
have  occurred  in  a  given  area  between  photographic  reconnaissance  missions 
sought  to  determine  whether  change  detection  in  image  interpretation  is  im¬ 
proved  by  (a)  demarcation  of  common  areas  in  early  and  late  imagery, 

(b)  annotation  of  targets  on  early  imagery,  and  (c)  use  of  a  list  of  tar¬ 
gets  appearing  on  the  early  imagery. 

Image  interpreters  (N  =  88)  were  given  a  change  detection  task  on  nine 
pairs  of  early  and  late  photographic  cover.  The  interpreter  subjects  were 
assigned  in  groups  of  11  to  each  of  8  treatment  combinations  of  the  3  vari¬ 
ables.  Performance  was  evaluated  in  terms  of  number  of  correct  change  de¬ 
tections  with  respect  to  "unchanged,"  "gone,"  and  "new"  targets,  number  of 
erroneous  detections,  and  total  working  time.  Principal  findings  of  the 
study  show  that  (a)  the  presence  or  absence  of  common  area  demarcation  had 
no  significant  effect  upon  the  mean  number  of  correct  change  detections  in 
any  of  the  three  change  categories;  (b)  providing  the  interpreter  with  tar¬ 
get  information  in  the  early  cover  significantly  increased  mean  number  of 
correct  change  detections  for  "unchanged"  and  "gone"  targets;  (c)  common 
area  demarcation  did  not  decrease  the  mean  number  of  erroneous  change  detec¬ 
tions;  (d)  use  of  target  lists  also  increased  the  mean  number  of  correct 
detections,  but  at  the  same  time  increased  the  number  of  "false  alarms," 
and  increased  working  time  by  about  20  percent;  (e)  combined  use  of  target 
annotations  and  target  lists  to  provide  information  about  early  cover  imagery 
appears  to  maximize  the  number  of  correct  change  detections. 


Kause,  R. ,  Thomas,  J.  A.,  and  Jeffrey,  T.  E.  Coordinate  determination  of 

SLAR  imaged  features.  Technical  Research  Note  234,  April  1973. 

This  research  project  was  conducted  to  improve  the  performance  of  op¬ 
erators  in  interpreting  SLAR  (side- looking  airborne  radar)  imagery  in  real¬ 
time  or  near  real-time  inflight  display  systems.  The  principal  objective 
of  this  experiment  was  to  determine  the  precision  with  which  image  inter¬ 
preters  can  determine  the  coordinate  location  of  targets  detected  on  a  near 
real-time  SLAR  display  or  in  a  simulated  situation. 

In  the  experiment,  22  graduate  enlisted  interpreters  performed  in  small 
groups  (no  more  than  6)  as  airborne  sensor  operators.  After  about  4  hours 
spent  in  a  review  on  radar  interpretation  and  the  use  of  GEO  (geographic) 
and  UTM  (Universal  Transverse  Mercator)  grid  systems,  the  men  were  given  2 
tasks  under  4  treatment  conditions.  The  tasks  were  (a)  to  identify  and 
provide  location  coordinates  for  targets  annotated  on  the  imagery  (directed 
search)  and  (b)  to  detect,  identify,  and  locate  tactical  targets  on  an  un¬ 
annotated  imagery  (free  search) .  Each  task  was  performed  using  imagery 
acquired  by  a  noncoherent  radar  system  AN/APS-94  and  again  using  imagery 
acquired  by  a  coherent  system  AN/APQ-102A.  other  factors  varied  were 
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topographic  maps  versus  radar  mosaics  for  determining  ground  location,  and 
UTM  versus  GEO  coordinates  for  reporting  location.  Performance  was  evalu¬ 
ated  in  terms  of  accuracy  and  completeness  in  target  location,  in  target 
detection,  and  in  target  identification. 

I'|  Neither  directed  not  free  search  resulted  in  location  data  that  met 
general  operational  requirements  for  accuracy.  In  both  directed  and  free 
search,  accuracy  and  completeness  of  target  identification  was  signifi¬ 
cantly  better  with  noncoherent  imagery.  In  free  search  no  differences  ob¬ 
tained  across  sensors  for  accuracy  and  completeness  in  target  detection. 

Over  the  entire  range  of  location  errors, . the  median  error  for  those  re¬ 
porting  GEO  coordinates  was  4,600  compared  to  5,900  for  those  reporting  UTM 
coordinates.  .  Findings  point  to  need  for  development  of  improved  operator 
techniques  so  that  targets  detected  in  SLAR  imagery  can  be  more  accurately 
located. 

Kause,  R. ,  Thomas,  J.  A.,  and  Jeffrey,  T.  E.  Effect  of  training  on  coordi¬ 
nate  determination  of  SLAR  imaged  features.  Technical  Research  Note 

235,  April  1973.  (AD  762  342)  ~  . . . 

\ 

This  research  developed  and  evaluated  a  training  program  to  improve 
the  performance  of  operators  in  locating  targets  in  side-looking  airborne 
radar  (SLAR)  imagery.  This  study  developed  a  trainee-participation  instruc¬ 
tional  program  with  immediate  feedback  and  assessed  its  effectiveness  in 
enhancing  image  interpreter  performance  with  respect  to  accuracy  in  locat¬ 
ing  and  identifying  map  coordinates  of  detected  targets. 

A  four-phase  instructional  unit  was  developed  varying  in  imagery  and 
map  scale,  orientation  cues,  and  presentation  order.  The  experimental 
subject  group  (12  school-trained  interpreters)  performed  two  tasks  involving 
detection,  identification,  and  determination  of  location  data  for  SLAR  imagery1 
targets  under  conditions  of  directed  search  and  free  search.  Performance  of 
the  experimental  group  was  compared  to  that  of  a  control  group  who  performed 
the  same  tasks  under  the  same  conditions  but  who  had  received  no  individual¬ 
ized  pretest  training. 

The  use  of  a  systematic  training  procedure  coupled  with  immediate  feed¬ 
back  (knowledge  of  results)  yielded  improved  performance  in  the  determination 
of  coordinate  location  of  tactical  targets  annotated  upon  SLAR  imagery.  The 
experimental  group  determined  location  data  for  targets  with  greater  pre¬ 
cision  than  did  the  control  group.  Accuracy  scores  were  lower  for  the  in¬ 
terpreters  who  had  been  given  specialized  training  than  for  the  control 
group.  Absolute  magnitude  of  location  errors  in  both  groups  exceed  that 
specified  as  operationally  acceptable.  This  aspect  may  be  particularly 
critical  when  considering  the  use  of  small-scale  SLAR  imagery  and  a  rela¬ 
tively  small-scale  topographic  map  (1:250,000).  An  improved  version  of  the 
experimental  instructional  unit  developed  should  prove  useful  in  increasing 
interpreter  effectiveness  in  correlating  SLAR  imaged  positions  with  map 
locations . 
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Jeffrey,  T.  E.  Effect  of  photo  degradation  on  interpreter  performance. 

Technical  Research  Note  245,  June  1973.  (AD  763  908) 

Scale,  haze,  and  image  motion  were  judged  to  be  the  key  factors  in  de¬ 
termining  the  interpretability  of  photographs  encountered  operationally, 
afid  images  varying  in  these  factors  were  produced  by  manipulating  opera- 
tipnal  imagery  in  the  laboratory.  Image  interpreters  were  given  tasks  of 
detecting  and  identifying  military  targets  on  these  images.  Their  scores 
were  used  to  assess  interpretability  as  a  function  of  scale ,  haze ,  and 
image  motion. 

j.  Results  of  the  experiment  showed:  (a)  When  considered  singly,  scale, 
haze,  and  image  motion  had  little  effect  on  interpreter  target  detection 
performance.  When  two  or  more  of  these  sources  of  degradation  were  present 
simultaneously,  target  detection  performance  deteriorated  markedly. 

(b)  In  general,  the  decrease  in  target  detection  accuracy  obtained  with 
increased  photo  degradation  appears  to  be  due  more  to  erroneous  classifica¬ 
tion  of  nontargets  as  targets  than  to  the  misclassification  of  targets  as 
nontargets,  (c)  Target  identification  accuracy  and  completeness  were  sig¬ 
nificantly  reduced  by  either  unidimensional  or  multidimensional  degrading 
conditions  of  the  type  used  in  this  experiment,  (d)  When  photographic  scale 
was  small,  the  effect  of  other  sources  of  degradation  on  interpreter  per¬ 
formance  was  significantly  greater  than  when  photographic  scale  was  large. 

The  findings  provide  direction  for  a  continuing  search  for  improved 
techniques  for  predicting  the  utility  of  aerial  reconnaissance  photographic 
.missions  and  for  guiding  the  establishment  of  mission  requirements. 


lowers,  J.  R. ,  III,  Brainard,  R.  W. ,  Abram,  R.  E. ,  and  Sadacca,  R.  Train¬ 
ing  techniques  for  rapid  target  detection.  Technical  Paper  242, 
September  1973.  (AD  768  194) 

Basic  to  the  acquisition  of  intelligence  information  from  aerial  sur¬ 
veillance/reconnaissance  sensor  products  is  the  ability  of  the  image  inter¬ 
preter  to  detect  (distinguish  objects  from  their  background)  and  correctly 
identify  targets  on  the  product  imagery.  This  publication  reports  on  the 
evaluation  of  two  systematic  search  strategies  and  a  "speed-reading"  tech¬ 
nique  for  use  in  training  interpreters  to  search  imagery  more  rapidly  and 
to  improve  detection  accuracy  and  completeness . 

Systematic  search  strategies  for  increasing  detection  completeness  and 
for  reducing  inefficient  search  behavior  were  (a)  geometric  (structured  prac¬ 
tice  in  executing  a  geometric  search  pattern)  and  (b)  tactical  (structured 
practice  in  executing  a  content-oriented  search  pattern) .  To  complement  the 
search  strategies,  a  "speed-reading"  technique  was  developed  to  reduce  visual 
fixation  time  and  expand  the  size  of  the  perceptual  field,  through  "speeded 
search"— practice  under  free  search  conditions  with  emphasis  on  speeded  per¬ 
formance.  A  fourth  and  control  condition  allowed  practice  under  free  search 
conditions  without  emphasis  on  speeded  performance.  Eight  subjects  were 
assigned  to  each  of  the  four  experimental  conditions,  and  their  pre-  and 
posttraining  target  detection  performances  were  compared.  An  error-avoidance 
training  technique  was  also  tried  out. 
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Training  in  the  two  strategies  increased  the  number  of  target  detec¬ 
tions  over  pretraining  levels  and  also  increased  the  number  of  inventive 
errors.  The  speeded  search  group  made  the  greatest  gain  in  image  search 
speed,  close  to  200%,  which  was  achieved  without  loss  of  accuracy  of  com¬ 
pleteness.  The  separately  studied  error  avoidance  training  technique, 
using  an  error  key  approach,  substantially  reduced  inventive  errors.  Er- 
roi;  avoidance  training  used  with  training  in  rapid  systematic  or  free 
search  appears  helpful  in  improving  detection  performance. 


Clarke,  F.  R. ,  Welch,  R.  I.,  and  Jeffrey,  T.  E.  Development  of  a  psycho¬ 
physical  photo  quality  measure.  Research  Report  1178,  January  1974. 

(AD  776  369) 

In  extracting  information  from  aerial  reconnaissance/sensor  products, 
the  image  interpreter  must  often  work  with  imagery  of  less  than  ideal 
quality.  The  Surveillance  Systems  research  program  of  the  Army  Research 
Institute  is  designed  to  identify  factors  of  photo  quality  that  can  help 
measure  the  interpretability  of  operational  imagery.  In  the  present  ex¬ 
periment,  image  scale,  atmospheric  haze,  and  image  motion  were  judged  to  be 
key  factors  in  operational  interpretability  of  imagery.  Laboratory  manipu¬ 
lation  of  representative  operational  images  produced  experimental  images 
with  controlled  degradation  of  these  three  factors,  separately  and  m  com¬ 
bination.  Image  interpreters  were  asked  to  detect  and  identify  military 
targets  on  this  degraded  imagery.  Their  scores  were  used  to  assess  inter¬ 
pretability  as  a  function  of  scale,  haze,  and  image  motion.  In  a  parallel 
experiment,  geometric  shapes  were  positioned  on  background  panels  of  black, 
white,  and  grey.  Photographs  of  this  array  were  degraded  with  respect  to 
scale,  haze,  and  image  motion  to  the  same  levels  as  the  first  experimental 
imagery,  and  similar  tests  were  conducted. 

Degradation  of  image  quality  by  changes  in  scale,  haze,  or  image  motion, 
taken  singly,  had  little  effect  on  interpreter  performance  in  target  detec¬ 
tion  or  identification.  When  more  than  one  factor  was  degraded  in  the  same 
image,  however,  significant  degradation  in  performance  was  noted. 

A  simple  model  based  on  ground-resolved  distance,  a  variable  combining 
the  effects  of  scale,  haze,  and  image  motion,  effectively  predicted  inter¬ 
preter  performance  for  actual  aerial  imagery  and  for  the  parallel  tests 
using  fabricated  geometric  target  shapes  on  constructed  model  imagery.  A 
prototype  catalog  of  image  quality  previously  developed  by  ARI  with  variants 
in  scale,  sharpness,  and  scene  complexity  also  predicted  the  interpretability 
of  the  aerial  imagery  used . 

Analytical  models  such  as  these  can  measure  the  interpretability  of 
aerial  reconnaissance  imagery,  and  thus  aid  in  selecting  the  more  inter¬ 
pretable  frames  and  enabling  skilled  interpreters  to  extract  intelligence 
information  most  efficiently.  The  fabricated  imagery  arrays  are  usefal  as 
low-cost  research  tools  for  identifying  image  quality  factors  independently 
of  scene  content. 
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Levine,  J.  M, ,  and  Eldredge,  D.  Effects  of  ancillary  information  upon 

photointerpreter  performance.  Technical  Paper  255,  September  1974. 

(AD  785  706) 

This  report  evaluates  the  improvement  of  photointerpretation  perfor¬ 
mance  achieved  by  providing  ancillary  intelligence  information  to  the  photo¬ 
interpreter.  The  research  is  presented  in  two  sections:  a  reprint  from 
Human  Factors,  1972,  14(6) ,  549-560?  and  an  appendix  that  supplements  the 
journal  article  with  additional  detail  on  the  photointerpretation  task 
studied  and  the  methodology  employed. 

Thirty-two  trained  surveillance  officers  identified  single  annotated 
targets  within  controlled  and  matched  sets  of  imagery.  For  each  annotation, 
the  subjects'  task  was  to  provide  a  vector  of  subjective  probabilities,  with 
each  probability  representing  the  estimated  likelihood  that  a  particular 
target  name  was  the  correct  identifier  for  the  annotated  item. 

Half  the  subjects  were  provided  with  ancillary  intelligence  information 
at  the  time  they  evaluated  the  image  displays?  the  other  half  made  their 
evaluations  without  information,  then  made  a  second  and  revised  evaluation 
using  the  ancillary  information.  Four  inodes  of  performance  were  examined: 
no  information,  simultaneous  information/image  interpretation,  postinforma¬ 
tion  (the  revised  evaluation) ,  and  computer  integration  (in  which  a  computer 
integrated  interpreter  no-information  evaluations  with  the  ancillary  infor¬ 
mation)  .  Half  the  information  supplied  was  qualitative  in  nature,  half 
quantitative?  information  variables,  which  included  target  difficulty,  credi¬ 
bility,  and  consistency,  were  arranged  so  that  their  influence  on  interpreter 
performance  could  be  assessed.  A  two  between-subject,  seven  within-subject 
factorial  research  design  was  employed. 

The  results  indicated  that  the  provision  of  ancillary  information  en¬ 
hanced  accuracy  of  identification  regardless  of  whether  the  information  was 
qualitative  or  quantitative,  and  regardless  of  whether  that  information  was 
presented  simultaneously  with  or  subsequent  to  the  imagery.  Furthermore, 
under  the  conditions  of  this  study,  the  integration  function  proved  to  be 
best  carried  out  by  the  interpreter,  not  by  computer. 


Root,  R.  T.,  Myers,  L.  B. ,  and  Narva,  M.  A.  Effects  of  acquisition  param¬ 
eters  on  interpretability  of  infrared  imagery  (U) .  Technical  Research 
Note  240,  December  1974.  (CONFIDENTIAL)  (AD  COOO  358) 

Infrared  sensing  systems  permit  the  acquisition  of  intelligence  infor¬ 
mation  under  an  extended  range  of  weather  conditions,  provide  round-the- 
clock  collection  capability,  and  permit  the  collection  of  information  not 
obtainable  with  other  systems.  This  research  investigated  the  influence  of 
various  factors  involved  in  the  acquisition  of  infrared  imagery  upon  the 
interpretability  of  the  imagery  and  to  determine  what  information  may  be 
obtained  under  various  conditions.  Inexperienced  interpreters,  after  a 
brief  training  course,  were  asked  to  interpret  sets  of  imagery  that  pre¬ 
sented  selected  combinations  of  acquisition  parameters  in  order  to  discover 
the  influence  of  each  parameter  on  interpretability.  The  results  can  be 
useful  in  planning  imagery  acquisition  missions. 
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Root,  R.  T. ,  Young,  R.  B. ,  and  Narva,  M.  A.  Characteristics  of  reference 
keys  for  use  in  the  interpretation  of  infrared  imagery  (U) .  Technical 
Paper  248,  December  1974.  (CONFIDENTIAL)  (AD  C000  538) 

ARI's  surveillance  systems  research  program  produces  scientific  data 
bearing  on  the  extraction  of  information  from  surveillance  displays,  and  on 
the  efficient  storage,  retrieval,  and  transmission  of  this  information  within 
an  advanced  computerized  image  interpretation  facility.  The  present  report 
deals  with  determining  the  most  effective  content  of  reference  keys  for  use 
in  interpreting  infrared  imagery. 


Cockrell,  J.  T.  Evaluation  of  four  target-identification  training  tech¬ 
niques.  Technical  Paper  301,  August  1978.  (AD  A061  175) 

As  part  of  a  research  program  seeking  more  efficient  methods  of  train¬ 
ing  image  interpreters  in  target  identification,  four  alternative  experi¬ 
mental  instructional  techniques  were  evaluated.  Eight  recently  trained 
image  interpreters  used  each  method  (N  *  32) .  Two  of  the  four  methods  used 
pictures  instead  of  text,  one  in  a  random  presentation  and  the  other  in  a 
structured  sequence  of  increasing  difficulty.  The  third  method  used  pro¬ 
gramed  text  to  teach  verbal  identification  cues,  and  the  fourth  combined 
programed  text  in  the  first  half  and  the  structured  pictorial  method  in  the 
last  half  of  training.  In  each  method,  half  the  students  received  feedback 
of  both  the  correct  answer  and  the  reason  a  wrong  answer  was  wrong,  and 
half  received  only  the  correct  answer. 

Students  were  evenly  divided  across  all  conditions  by  their  general 
technical  (GT)  aptitude  score:  high  (at  or  above  124)  and  low  (below  124) . 

Posttests  on  visual  target  identification  and  verbal  target  cues  evalu¬ 
ated  the  effectiveness  of  the  methods  and  feedback  conditions  for  each  GT 
level . 


Target  identification  performance  was  significantly  poorer  with  the 
structured  text  method  but  about  the  same  among  the  three  methods  using 
pictures.  Recognition  of  target  cues  was  significantly  better  when  verbal 
instructions  on  target  cues  were  given  than  when  training  was  entirely  pic¬ 
torial.  Type  of  feedback  had  no  significant  effect.  Learning  performance 
did  not  differ  as  a  function  of  GT  aptitude,  but  interpreters  with  lower 
aptitude  forgot  their  training  more  rapidly. 


Lepkowski,  J.  R.  Image  interpreter  screening  performance  as  affected  by 
resolution,  presentation  rate,  and  scale.  Technical  Paper  335,  Sep¬ 
tember  1978.  (AD  A064  262) 


This  study  measured  performance  levels  of  image  interpreters,  using 
both  direct  viewing  and  viewing  with  magnification,  to  screen  70mm  photo¬ 
graphic  imagery  moving  at  six  different  rates  and  varying  in  scale  and 
resolution.  The  resolution  of  the  imagery  was  varied  to  simulate  imagery 
obtained  with  different  transmission  bandwidths.  The  results  were  as 
follows: 
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1.  There  was  little  loss  in  screening  accuracy  at  any  level  of  reso¬ 
lution  and/or  magnification  for  frame  presentation  rates  of  6.0 
to  1.5  sec/frame  with  the  Is 2, 000  scale  imagery  and  6.0  to  2.0 
sec/frame  for  the  Is 4,000  scale  imagery. 

2.  In  general,  the  poorer  the  image  resolution,  the  lower  the  screen¬ 
ing  accuracy. 

3.  For  constant  display  format  and  image  resolution,  screening  accu¬ 
racy  was  significantly  greater  for  the  larger  scale  display. 

4.  Two-power  optical  magnification  of  the  photographic  imagery  re¬ 
sulted  in  a  decrease  in  screening  accuracy.  Increasing  viewing 
time  for  magnified  imagery  by  a  factor  of  four  so  that  the  incre¬ 
mental  area  viewed  per  second  was  equated  for  direct  and  magni¬ 
fied  viewing  resulted  in  no  significant  difference  between  the 
two  viewing  conditions. 

Tradeoffs  are  possible  between  and  among  the  factors  of  scale,  presen¬ 
tation  rate,  and  resolution. 


Sewell,  E.,  Bradie,  R. ,  Harabedian,  A.,  and  Jeffrey,  T.  The  effects  of 

photo  characteristics  upon  location  determination  in  a  photogrammetric 
facility.  Technical  Paper  346,  October  1978.  (AD  A062  255) 

The  operator  of  the  Army  Analytical  Photogrammetric  Positioning  System 
(APPS)  must  perform  a  critical  step  in  the  coordinate  determination  procedure 
by  the  subjective  comparison  of  dissimilar  images.  He  must  correlate  recon¬ 
naissance  mission  imagery  with  aerotriangulated  photographic  stereoscopic 
pairs,  known  as  data  base  imagery,  and  visually  identify  the  image  position 
on  t  e  data  base  of  a  target  detected  on  the  mission  image.  Results  obtained 
from  40  image  interpreters  performing  this  task  using  vertical,  oblique,  and 
panoramic  mission  imagery  are  given.  Two  levels  of  target  position  diffi¬ 
culty  were  established:  (a)  A-points  located  at  terrain  or  man-made  features 
mutually  identifiable  on  both  mission  and  data  base  imagery;  (b)  B-points 
remote  from  terrain  or  man-made  features  mutually  identifiable  on  mission 
and  data  base  imagery. 

Transfer  of  A-points  for  vertical,  oblique,  and  panoramic  photographs 
was  accomplished  with  good  accuracy — median  location  error  was  19  meters  or 
less.  Performance  was  signif icantly  better  when  A-points  in  panoramic  mis¬ 
sions  were  located  in  the  near-vertical  half  of  the  image  in  contrast  to  the 
near-horizon  half.  Similarly,  panoramic  mission  imagery  in  film  transparency 
form  resulted  in  more  accurate  transfer  than  when  presented  as  an  opaque 
paper  print. 

Transfer  of  B-points  war  more  difficult.  For  vertical  missions,  median 
location  error  was  about  20  meters.  For  oblique  or  panoramic  imagery,  lo¬ 
cation  errors  were  markedly  greater  when  the  B-points  were  located  in  the 
horizon  half  of  the  image  as  compare^  to  performance  when  the  points  were 
in  the  near-vertical  half  of  the  image.  B-points  in  the  horizon  half  of  low 
panoramic  missions  were  more  accurately  transferred  when  in  film  transparent 
format  than  when  displayed  on  opaque  paper  prints. 
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Sewell,  E.,  Harabedian,  A.,  and  Jeffrey,  T.  Mission/data-base  imagery  cor¬ 
relation  techniques  (M/DICT) .  Technical  Paper  347,  October  1978, 

(AD  A064  264) 

The  Analytical  Photogrammetric  Positioning  System  (APPS)  was  developed 
to  determine  accurate  target  location  data.  APPS  operators  transfer  target 
lo6ations  from  mission  images  to  a  vertical  data-base  photograph  in  deter¬ 
mining  target  coordinates. 

The  speed  and  accuracy  with  which  experienced  interpreters  of  side- 
looking  radar,  infrared,  and  television  images  transferred  annotated  loca¬ 
tions  from  such  sensor  images  to  a  vertical  data-base  photograph  were  deter¬ 
mined.  Annotated  locations  used  were  A-type  (those  on  or  near  a  terrain 
feature  identifiable  on  mission  and  data-base  images)  and  B-type  (those  200 
or  more  meters  distant,  on  the  ground,  from  such  mutually  identifiable  fea¬ 
tures)  .  Transfer  techniques  were  of  two  kinds:  (a)  direct  transfer  in 
which  visual  correlation  of  mission  and  data-base  images  was  used:  (b)  in¬ 
direct  transfer  where  the  required  point  was  transferred  using  a  photogram¬ 
metric  transformation  method  employing  auxiliary  points  that  were  mutually 
identifiable  on  mission  and  data-base  images. 

Results  from  nine  interpreters  are  summarized  below: 

Proportion  of  Transfers  by  Error  Range  (Meters) 


Transfer 

A-type  locations 

B-type  locations 

technique 

25%  50%  75% 

25%  50% 

75% 

Direct 

0-21  0-34  0-52 

0-39  0-75 

0-130 

Indirect 

Same  as  above 

0-35  0-54 

0-64 

The  improved  accuracy  of  the  indirect  method  for  B-type  locations  requires 
more  time.  Direct  transfer  required  about  1.3  minutes  per  target  on  the 
average  while  indirect  transfer  per  target  on  the  average  was  five  times 
that  amount  of  6.5  minutes  per  target. 


Sewell,  E.,  Harabedian,  A.,  and  Jeffrey,  T.  Total  system  accuracy  for  APPS 
(The  Analytical  Photogrammetric  Positioning  System) .  Technical  Paper 
348,  October  1978.  (AD  A063  595)  ”  ~ 

The  accuracy  and  speed  with  which  experienced  APPS  operators  can  obtain 
UTM  coordinates  and  the  elevation  above  the  datum  plane  for  points  of  inter¬ 
est  designated  on  various  reconnaissance  imagery  types  were  determined. 

These  points  of  interest  were  transferred  by  an  analytical  method  from  repre¬ 
sentative  operational  imagery— panoramic  photography,  infrared,  and  radar — 
to  1:100,000  scale  data  base  photography  and  their  UTM  coordinates  and  ele¬ 
vation  above  the  datum  plane  determined.  Results  were  compared  with  known 
values  and  error  magnitudes  and  performance  time  requirements  obtained.  Re¬ 
sults  indicate  that  experienced  APPS  operators  are  very  consistent  in  their 
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placement  of  points  of  interest  under  the  reference  dot  (monocular  pointing) . 
(NOTE:  All  distances  are  reported  at  ground  scale  and  not  at  the  distance 
measured  on  the  data  base  image.)  The  median  variability  error  was  1.6  meters 
in  the  XY-plane.  Time  required  per  point  for  monocular  pointing  was  about 
.18  minutes.  On  the  average,  stereoscopic  pointing— the  placement  of  the 
point  of  interest  in  the  three-dimensional  data  base  model  in  apparent  con¬ 
tact  with  the  floating  reference  dot— was  accomplished  with  a  median  eleva¬ 
tion  error  of  3  meters,  a  median  XY-plane  error  of  6  meters,  and  a  median 
total  error  of  7.5  meters.  Median  time  required  per  point  for  stereoscopic 
pointing  was  .8  minutes.  Finally,  the  transfer  of  a  point  of  interest  from 
a  reconnaissance  to  data  base  image  combined  with  stereoscopic  pointing  gave 
results  that  varied  with  the  type  of  reconnaissance  imagery  used.  For  infra¬ 
red  imagery,  median  elevation  error  was  5.7  meters  median  XY-plane  error 
was  9.5  meters,  and  median  total  error  was  12.3  me  !.  The  median  time  per 
point  to  transfer  and  obtain  coordinate  data  was  about  7  minutes.  Similar 
but  somewhat  larger  median  errors  were  obtained  for  panoramic  and  radar  re¬ 
connaissance  missions .  The  speed  and  accuracy  with  which  these  experienced 
APPS  operators  determined  coordinate  and  elevation  data  are  well  within  the 
initially  stipulated  requirements. 


Huntoon,  R.  B. ,  Schohan,  B. ,  and  Shvern,  U.  Visual  search  performance  in 
simulated  remotely  piloted  vehicle  utilization  as  a  function  of  aux¬ 
iliary  task  loading  on  the  observer.  Technical  Paper  357,  April  1979, 
(AD  A072  402) 

Baseline  data  were  obtained  on  how  well  observers  could  extract  infor¬ 
mation  from  a  TV  monitor  while  performing  auxiliary  tasks  under  task  loading 
conditions  that  might  be  encountered  in  use  of  a  remotely  piloted  vehicle 
(RPV)  as  the  sensor  platform. 

A  simulation  facility  was  used.  It  contained  a  terrain  model,  a  TV 
camera  transport,  hybrid  computing  equipment,  and  a  television  display  and 
control  console. 

Six  pilots  and  six  nonpilots  participated  in  the  three-phase  effort. 
Phase  A  required  participants  to  detect  and  recognize  tank-sized  targets 
in  open  and  cluttered  backgrounds  from  a  simulated  altitude  of  2,000  feet 
and  a  simulated  RPV  velocity  of  100  knots .  Phase  B  required  participants 
to  monitor  and  correct  deviations  in  the  RPV  course  and  altitude  and  to 
respond  to  two  visual  warning  indicators.  Phase  B  tasks  were  presented 
at  two  rates:  one  per  10  seconds  and  three  per  10  seconds.  Phase  C  com¬ 
bined  the  tasks  of  Phase  A  and  Phase  B  with  concurrent  task  demands  upon 
the  participants. 

Increasing  the  auxiliary  load  level  decreased  the  probabilities  and 
ranges  of  target  detection  and  recognition.  Target  acquisition  task  de¬ 
mands  similarly  increased  auxiliary  task  response  times.  Cluttered  back¬ 
ground  significantly  degraded  target  acquisition  task  performance,  particu¬ 
larly  when  the  auxiliary  task  was  performed  concurrently. 

While  the  results  of  the  research  are  important  to  military  management, 
research  scientists  will  be  the  principal  readers  of  the  report. 
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Evans,  L.  A.,  and  Swenson,  R.  C.  Vectors  of  probable  accuracy  of  image 

interpreter  performance.  Working  Paper  HF-79-05,  June  1979, 

The  purpose  of  this  report  was  to  investigate  techniques  for  obtaining 
complete  probability  vectors  for  each  target  reported  by  an  image  inter¬ 
preter.  Three  general  techniques  were  compared.  Three  groups  of  trained 
Army  image  interpreters  were  used,  each  group  employing  one  of  the  three 
probability  vector  generating  techniques.  A  hierarchically  organized  target 
list  was  developed  for  the  interpreters'  reports.  Knowledge-of -results 
training  was  given  to  improve  the  interpreters'  ability  to  estimate  the 
probable  accuracy  of  their  reports.  Test  sessions  examined  transfer  of 
feedback  training  and  differences  in  judgment  techniques  to  a  free  search 
interpretation  task.  The  feasibility  of  using  multiple  regression  predic¬ 
tion  equations  to  improve  the  interpreters'  direct  estimates  of  their  prob¬ 
able  accuracy  was  also  investigated.  Of  the  techniques  investigated,  the 
direct  name  vector  approach  appeared  to  maximize  the  amount  of  information 
which  could  be  obtained.  Since  the  basis  on  which  interpreters  made  esti¬ 
mates  appeared  to  change  from  training  to  test  sessions ,  the  multiple  re¬ 
gression  adjustment  procedure  was  inconclusive. 


Laymon,  R.  S.  Studies  of  image  interpreter  estimates  of  unit  identifica¬ 
tion.  Working  Paper  79-06,  June  1979. 

Four  pilot  studies  were  conducted  to  obtain  information  as  to  how  well 
relatively  inexperienced  image  interpreters  could  estimate  probabilities, 
probability  vectors,  and  make  confidence  estimates  with  respect  to  large 
military  units  given  the  compositions  of  the  units  and  a  sample  of  targets 
from  one  of  the  units.  The  units  were  defined  as  company  or  battalion  level 
units  consisting  of  a  variety  of  vehicles.  The  sample  was  a  subset  of  ve¬ 
hicles  from  one  of  the  units.  Several  procedural  techniques  for  enhancing 
subject  performance  were  studied  as  well  as  various  characteristics  of  the 
samples  and  units.  By  and  large,  the  sample/unit  problems  used  in  these 
studies  were  too  difficult  for  the  subjects  to  provide  useful  information. 
This  result  indicates  that  for  the  types  of  units  used  in  these  studies, 
that  is,  company  and  battalion  level  units  with  sample  sizes  ranging  up  to 
75%  of  the  unit,  relatively  naive  image  interpreters  should  not  be  permitted 
to  make  such  judgments. 

The  results  of  Study  Four  do  provide  some  specific  data  on  certain 
sample/unit  characteristics  that  appear  to  enhance  or  degrade  performance. 
Further,  experience  in  operating  the  computer  program  developed  for  Study 
Four  indicates  that  the  tasks  imposed  by  these  studies  may  be  reasonable 
for  interpreters  who  receive  intensive  training  to  make  similar  judgments 
by  means  of  this  computer  program. 

Recommendations  for  further  research  are  discussed. 


Martinek,  H.,  and  Zarin,  A.  The  effects  of  bandwidth  compression  on  image 
interpreter  performance.  Technical  Report  396,  August  1979. 

(AD  A077  840) 

Conventional  imagery  differing  in  resolution  (8-inch  ground  resolved 
distance  (GRD) ,  16-inch  GFD,  and  24-inch  GRD) ,  digitized  and  treated  to 
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represent  four  levels  of  bandwidth  compression  (1:1  -  no  compression,  4:1, 
8:1,  and  10:1)  was  interpreted  by  12  experienced  image  interpreters.  A 
Greco- Latin  square  design  was  used  to  control  for  differences  between  in¬ 
terpreters,  imagery  subsets,  and  period  effects  at  each  3RD  and  to  test  the 
effects  of  bandwidth  compression.  The  dichotomized  image  variables  of  sun 
angle,  image  contrast,  and  target  obscurity  were  distributed  equally  across 
all  conditions,  but  could  be  analyzed  only  for  the  24-inch  GRD  imagery. 
Significant  decreases  were  found  in  the  number  of  correct  identifications 
due  to  bandwidth  compression;  the  largest  decrease  occurred  between  4:1 
and  8:1  bandwidth  compression  ratios.  System  developers  should  make  care¬ 
ful  trade-off  evaluations  in  using  bandwidth  compression.  More  research  in 
this  area  is  required  on  the  search  function  of  interpretation  and  interac¬ 
tions  of  image  variables. 


Montgomery,  C.  A,,  Thompson,  J.  R. ,  Katter,  R.  V.  Imagery  intelligence 

(IMINT)  production  model.  Research  Report  1210,  January  1980. 

(AD  A086  455) 

The  Imagery  Intelligence  (IMINT)  Production  Model  presented  in  this 
report  was  developed  as  a  part  of  a  study  entitled  "Investigation  of  Metho¬ 
dologies  and  Techniques  for  Intelligence  Analysis,"  wnich  is  aimed  at  con¬ 
structing  a  model  of  the  cognitive  processes  underlying  intelligence  analy¬ 
sis.  The  approach  to  constructing  such  a  model  is  based  on  the  investigation 
of  analytical  processing  in  two  types  of  single  source  analysis  (SIGINT  and 
IMINT),  subsequently  generalizing  to  multisource  analysis.  For  the  pur¬ 
poses  of  this  investigation,  intelligence  analysis  was  defined  as  a  spectrum 
of  analytical  judgmental  activities  involved  in  the  processing  and  production 
of  intelligence,  where  particular  individuals  may  devote  more  or  less  time 
to  different  aspects  of  such  activities  according  to  their  individual  roles 
in  the  intelligence  cycle. 

The  IMINT  production  model  described  in  this  report  was  developed  to 
serve  as  a  basis  for  selecting  IMINT  processes  involving  high  analytical  and 
judgmental  content  for  further  study,  directed  at  understanding  the  cogni¬ 
tive  functions  that  underlie  these  IMINT  processes.  Thus,  the  model  was 
constructed  with  a  specialized  focus ,  for  the  purposes  of  the  study  described 
above.  Rather  than  concentrating  exclusively  on  either  strategic  or  tactical 
IMINT,  the  model  was  designed  to  encompass  both  dimensions  of  IMINT.  The 
rationale  for  this  decision  was  the  fact  that  imagery  interpretation  analysts 
(IIAs)  with  more  than  minimum  length  of  service  are  likely  to  operate  in 
each  type  of  mission  at  some  time  in  their  career,  demonstrating  the  practi¬ 
cal  assumption  that  the  required  knowledge  base  and  cognitive  skills  are 
similar  for  the  two  dimensions  of  IMINT.  The  imagery  interpretation  analyst 
is  the  true  generic  focus  of  the  IMINT  production  model  presented  in  this 
report;  in  order  to  gather  data  for  developing  the  model  56  interviews  with 
IIAs  were  carried  out  at  8  sites,  including  both  strategic  and  tactical 
missions. 

The  report  contains  three  sections:  an  introduction,  an  overview  of 
the  IMINT  production  model,  and  the  model  description.  Section  1  describes 
the  IMINT  model  in  the  context  of  the  study  discussed  above,  and  also  treats 
the  scope  of  the  model  and  the  collection  of  the  information  on  which  the 
model  is  based.  Section  2  presents  an  overview  of  the  model,  describing 
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findings  about  the  role  of  the  imagery  interpretation  analyst,  the  impact 
of  management  on  analysis,  the  IMINT  time  dimension,  and  the  special  role 
of  informal  information  channels  in  IMINT  analysis.  This  section  also  de¬ 
scribes  the  format  of  the  model  description,  which  is  represented  in  terms 
of  hierarchical  input-process-output  (hipo)  charts.  Section  3  presents 
the  model,  which  is  segmented  into  two  major  parts:  IMINT  production  man¬ 
agement  activities  and  IMINT  processing,  analysis,  and  reporting  activities. 


Jeffrey,  T.  E.,  Martinek,  H.,  Shvern,  U.,  and  Johnson,  E.  M.  Summary  of  ARI 

image  interpretation  research.  Research  Report  1252 ,  July  1980. 

Image  interpretation  research  conducted  at  ARI  has  produced  scientific 
data  on  improving  the  extraction  of  information  from  surveillance  displays 
and  on  the  efficient  storage,  retrieval,  and  transmission  of  this  information. 
This  report  summarizes  research  on  image  interpretation  completed  by  ARI  be¬ 
tween  1970  and  1979,  organized  according  to  nine  major  problem  areas.  The 
rationale  of  ARI's  approach  to  the  problem,  findings,  operational  applications, 
and  further  research  requirements  are  presented  for  each  area.  The  research 
areas  cover  image  interpretability ,  real-time  and  near  real-time  imagery 
interpretation,  man/computer  decision  processes,  change  detection,  mensura¬ 
tion  and  coordinate  determination,  training  and  proficiency  maintenance,  key 
development,  and  reconnaissance  resource  management  and  utilization.  Con¬ 
tinued  utilization  of  these  research  findings  can  enhance  the  performance  of 
the  human  component  in  current  systems  as  well  as  provide  information  for 
system  developers  to  help  them  provide  design  specifications  for  future  sys¬ 
tems  and  to  determine  areas  needing  further  research. 


Ray,  T.  E,,  King,  R.  B.,  and  Narva,  M.  A.  Experimental  investigation  of 

near  real-time  interpretation  techniques  for  transmitted  imagery  (U) . 

Research  Note  80-24,  August  1980. 

Eight  interpretation  concepts  such  as  might  be  utilized  at  a  ground 
terminal  in  conjunction  with  the  handling  of  transmitted  imagery  were 
simulated.  Four  were  one-man  concepts  while  the  other  four  utilized  two- 
man  teams  viewing  the  imagery  in  sequence.  The  one-man  concepts  were  formu¬ 
lated  around  different  combinations  of  the  elements  of  availability  of  film 
speed  control,  viewing  area,  and  methods  of  target  designation  and  location 
reporting.  The  two-man  concepts  differed  on  the  availability  of  speed  con¬ 
trol  and  the  decision  criteria  for  the  initial  man  of  the  team  and  the  as¬ 
sociated  rescreening  strategy  of  the  second  man  of  the  team.  Military  image 
interpreters  detected,  identified,  and  reported  targets  on  the  imagery, 
presented  on  motorized  light  table  under  two  film  input  rates,  utilizing 
the  conditions  of  the  various  experimental  interpretation  concepts.  Inter¬ 
pretation  performance  was  found  to  be  influenced  by  the  size  of  the  viewing 
area  and  the  use  of  a  magnifying  reticle  to  localize  targets,  in  the  one- 
man  concepts.  The  incorporation  of  the  speed  control  option  did  not  sig¬ 
nificantly  affect  performance.  Provision  for  control  of  film  movement  or 
differential  emphasis  on  accuracy  or  completeness  had  no  significant  effects 
in  the  two-man  concepts. 
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Root,  R.  T.,  Ray,  T.  E.,  Brahosky,  A.  E.,  and  Narva,  M.  A.  A  study  of  the 
design  and  utilization  of  an  infrared  data  base  for  an  advanced  image 
interpretation  facility.  Research  Note  80-2S,  August  1980. 

A  reference  information  data  base  for  use  in  an  advanced  image  inter¬ 
pretation  facility  was  designed.  The  data  base  utilized  three  types  of 
organization.  Two  types  of  organization  utilized  primarily  representative 
infrared  imagery  examples.  One  type  presented  views  of  several  targets 
under  one  set  of  acquisition  parameters  while  the  second  type  presented 
views  of  one  target  type  under  all  of  the  acquisition  conditions.  The  third 
type  of  organization  utilized  primarily  textual  information.  An  indexing 
scheme  and  retrieval  methodology  were  devised.  The  experimental  data  base, 
with  associated  software,  was  utilized  to  subject-developed  concepts  to 
empirical  test.  One  test  was  designed  to  ascertain  if  structured  exposure 
to  the  data  base  would  increase  interpreter  proficiency  while  another  test 
was  designed  to  study  the  efficiency  of  such  a  data  base  as  an  aid  during 
interpretat ion . 

It  was  found  that  the  request  formats  used  on  the  CRT  computer  inter¬ 
face  could  be  efficiently  utilized  with  little  training.  The  information 
presented  on  the  slides  in  the  data  base  could  be  easily  utilized  for  train¬ 
ing  and  as  reference  keys.  Structured  exposure  to  the  data  base  in  training 
sessions  served  to  increase  the  participants'  proficiency  in  identification, 
at  the  category  level.  However,  no  differences  were  found  in  performance 
between  the  two  organizations  of  the  imagery  in  the  data  base. 


King,  R.  B. ,  Cooper,  I.,  and  Jeffrey,  T.  E.  Real  time  infrared  interpreta¬ 
tion  in  the  Mohawk  (U) .  Research  Note  80-26,  September  1980. 
(CONFIDENTIAL) 

This  research  had  two  objectives:  (a)  to  obtain  estimates  for  the 
levels  of  infrared  interpretation  performance  that  an  operational  airborne 
sensor  operator  might  attain  in  extracting  intelligence  information  from 
the  Mohawk,  OV-ID,  inflight  display?  and  (b)  to  examine  the  relative  ef¬ 
fects  of  selected  display  and  mission  variables  on  operator  performance. 

To  accomplish  these  objectives,  three  levels  of  aircraft  velocity  and  three 
levels  of  field-of-view  (FOV)  were  experimentally  manipulated  under  condi¬ 
tions  simulating  line,  area,  and  point  surveillance  missions.  The  effect 
of  utilizing  the  frame-hold  capability  inherent  in  the  display  was  investi¬ 
gated  in  conjunction  with  the  simulated  point  surveillance  condition. 


Special  Reports 

Key  to  U.S.  Army  tents,  1969. 

ARI  designed  and  developed  an  image  interpretation  key  to  Army  tents 
(Martinek,  Bigelow,  &  Jorgenson,  1968) .  It  was  prepared  specifically  for 
the  use  of  ARI  image  interpreters  in  preparing  scoring  keys  for  research 
purposes  and  for  Army  and  other  military  personnel  participating  in  image 
interpretation  research.  It  was  intended  to  help  interpreters  identify 
various  types  of  tents  rapidly  and  accurately. 
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^he  key  included  most  of  the  standard  Army  tents  then  in  common  use 
(September  1966) .  A  faw  types  were  obsolete  and  were  so  designated.  They 
were  included  because  they  still  occurred  in  isolated  instances  on  recent 
photographs  of  Army  camps  and,  more  frequently,  on  earlier  photo  coverage. 

Operational  Applications .  The  Army  tent  key  can  be  used  as  a  research 
tool  to  help  identify  tents  in  the  determination  of  "image  truth"  for  scor¬ 
ing  purposes  by  experimental  subjects  in  responding  to  test  materials  in¬ 
volving  U.S.  Army  equipment,  and  in  training  exercises  in  the  field. 


Target  reporting  terminology  and  target  recognition  key  (JTF-2  test  4.4)  , 

1969. 

Joint  Task  Force  Two  (JTF-2)  was  organized  by  the  Joint  Chiefs  of  Staff 
to  conduct  a  series  of  coordinated  and  integrated  tests  to  determine  the 
capabilities  and  vulnerabilities  of  offensive  and  defensive  reconnaissance 
systems  in  the  low-altitude  flight  regime.  Among  other  objectives,  Test  4.4 
of  the  test  program  was  designed  to  measure  the  target  acquisition  capa¬ 
bilities  of  both  the  aerial  observer  and  image  interpretation  systems  in 
representative  aircraft  systems  under  visual  flight  conditions  over  rolling 
terrain. 

ARI  was  requested  to  develop  material  to  standardize  reporting  termi¬ 
nology  and  to  acquaint  aircrews  and  image  interpreters  with  the  character¬ 
istics  of  the  different  target  complexes.  ARI.  prepared  a  booklet  that  con¬ 
tained  a  Target  Reporting  Terminology  List  (section  A)  and  a  Target 
Recognition  Key  (section  B)  (Army  Research  Institute,  1969).  The  Target 
Reporting  Terminology  List  specified  the  names  of  the  various  target  com¬ 
plexes  and  identified  the  vehicular  and/or  equipment  content  of  each  target 
complex.  The  Target  Recognition  Key  presented  these  same  target  complexes 
in  oblique  and  vertical  views  accompanied  by  annotation  of  target  items  in 
each  complex,  descriptions  of  the  vehicles  and  items  of  equipment  shown, 
description  of  the  target  environment,  and  factors  useful  for  making  identi¬ 
fications.  These  booklets  must  have  been  useful  to  the  aircrews  partici¬ 
pating  in  Test  4.4  because  few  were  returned  after  the  test. 

Operational  Applications .  The  material  developed  was  tailored  for  the 
JTF-2  field  evaluation  and  is  not  directly  apnlicable  to  other  situations. 

The  technique  employed  is  general  and  would  be  useful  as  a  way  to  standardize 
aerial  observer  (and  other)  reports  and  to  train  aerial  observers  to  recognize 
targets  and  items  of  equipment  in  other  situations. 


U.S.  Equipment  Mini-Key,  1969. 

An  unpublished  key  (ARI)  termed  the  U.S.  Equipment  Mini-Key  is  a  two- 
page  photographic  key  produced  by  ARI.  Scale  model  vehicles  (HO  gauge 
1:80)  and  items  of  U.S.  equipment  were  arrayed  on  a  terrain  board  and  photo¬ 
graphed  in  vertical  and  oblique  views.  The  objects  in  the  arrays  were  tic 
marked  and  numbered.  Target  names  and  dimensions  are  given  on  the  lower  half 
of  the  vertical  imagery. 
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This  key  was  developed  for  use  in  research  by  interpreters  participat¬ 
ing  in  experiments.  It  provides  a  convenient  method  of  assisting  inter¬ 
preters  to  identify  various  objects  of  U.S.  equipment.  Several  operational 
units  (in  the  Air  Force  and  Army)  have  found  this  key  to  be  useful  for 
training  and  maneuvers.  The  two  pages  of  the  Mini-Key  appear  as  Figure  5 
and  Figure  6. 

Operational  Applications .  The  Mini-Key  provides  a  useful  and  conven¬ 
ient  guide  for  identifying  Army  equipment  for  research  and  operational  use. 
Several  operational  units  use  the  key  for  training  exercises. 

Mini-Keys  of  potential  enemy  equipment  should  be  developed  for  training 
and  operational  use. 

Research  Recommendations.  None. 


Exercises  for  error  avoidance  training  in  image  interpretation,  1971. 

An  unpublished  instructional  manuscript  (Army  Research  Institute,  1971) 
was  developed  as  a  training  package  to  familiarize  students  with  the  use  of 
image  interpretation  keys  to  detect  common  errors  made  in  identifying  ob¬ 
jects  on  aerial  photographs.  Two  distinct  image  interpretation  keys  were 
presented.  The  first  document,  Error  Keys  as  Reference  Aids,  provided  the 
explanatory  material  for  photographic  imagery  acquired  in  France  and  Belgium 
during  World  War  II  and  had  been  used  in  an  earlier  error  key  research 
(Birnbaum,  Saciacca,  Andrews,  &  Narva,  1969).  This  text  material  provides  a 
step-by-step  set  of  instructions  to  the  student,  explaining  the  various 
error  types,  such  as  tree  shadow  error,  size  error,  tree  top  error,  and  so 
forth.  Annotated  loc-  -.ions  on  the  imagery  used  with  the  instructions  are 
specified. 

The  second  document,  Interpretation  Errors  on  Vietnam  Imagery,  paral¬ 
lels  the  first  for  a  different  geographic  area  and  for  another  culture.  In 
addition  to  tree  errors  and  size  errors,  probably  the  largest  number  of 
errors  made  on  Vietnam  imagery  were  caused  by  misinterpreting  local  religious 
and  agricultural  features  as  military  targets.  Shrines,  temples,  walled 
graves,  and  grave  mounds  appear  frequently  and  are  often  misinterpreted. 

In  the  first  document,  the  student  was  given  a  systematic  tour  through  the 
various  types  of  errors  frequently  made  by  peers.  In  addition,  the  second 
document  emphasized  common  emissions  made  by  interpreters  of  targets  of 
military  significance  in  Vietnamese  imagery. 

Operational  Applications.  These  manuscripts  provide  a  point  of  de¬ 
parture  for  the  development  of  a  training  unit  on  error  avoidance  for  formal 
training  of  image  interpreters  and/or  for  on-the-job  training. 

Research  Recommendations.  Work  should  be  extended  on  the  development 
of  expanded  training  units  for  error  avoidance. 
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Castelnovo,  A.  E.,  Tiedemann,  J.  G.,  and  Dobbins,  D.  A.  Performance  of 

single  vs.  multiple  voice  radio  transcribers  working  under  three  speech 
to  noise  ratios.  Technical  Research  Note  135,  September  1963. 

The  accurate  reception  of  a  radio  message  when  the  voice  is  masked  by 
atmospheric  noise  remains  a  serious  problem  in  radio  communication.  In 
research  to  improve  the  output  of  the  Army's  voice-radio  communication  sys¬ 
tems,  a  study  was  conducted  to  explore  the  possibility  that  a  transcription 
method  based  on  the  efforts  of  two  or  more  operators  would  result  in  fewer 
errors  and  a  more  complete  transcription  than  would  efforts  of  a  single 
transcriber . 

Three  work  methods  differing  in  the  number  of  transcribers  contributing 
to  the  final  product  were  compared  with  respect  to  accuracy  and  completeness 
of  the  resulting  transcription.  These  methods  (involving  one,  two,  and  three 
transcribers)  were  tested  under  three  levels  of  noise  which  yielded  high, 
medium,  and  low  intelligibility  under  standard  conditions.  Data  obtained 
were  interpreted  by  analysis  of  variance  techniques. 

A  small  but  consistent  difference  in  favor  of  multiple  transcriber  work 
methods  was  found.  While  differences  were  small  (significant  only  at  the 
medium  noise  level) ,  the  additional  amount  of  material  correctly  transcribed 
might  be  of  sufficient  importance  in  critical  situations  to  warrant  use  of 
more  than  one  transcriber .  Repeated  exposure  to  a  given  message  resulted 
in  some  increase  in  accuracy  at  low  and  medium  noise  levels.  At  high  noise 
level,  repeated  exposures  were  conducive  to  error. 


Castelnovo,  A.  E.,  Tiedemann,  J.  G.,  and  Skordahl,  D.  M.  Individual  dif¬ 
ferences  in  transcribing  voice  radio  messages  embedded  in  atmospheric 
noise.  Technical  Research  Note  137,  October  1963. 

Getting  accurate  and  complete  versions  of  a  radio  message  when  the  voice 
is  masked  by  atmospheric  noise  remains  a  serious  problem  in  radio-telephone 
communication.  Do  individuals  show  consistent  differences  in  accuracy  and 
completeness  of  transcription  of  voice  radio  messages?  Are  any  such  differ¬ 
ences  predictable  by  conventional  measures? 

Voice-radio  transcriber  performance  under  low,  medium,  and  high  levels 
of  atmospheric  noise  and  for  six  daily  3-hour  sessions  was  analyzed  for 
stability  of  performance  over  time  and  across  noise  levels.  Results  of  the 
small  sample  of  50  trainees  indicated  pronounced  and  reliable  individual 
differences  in  completeness  and  accuracy  of  the  transcripts  produced.  Ability 
to  minimize  the  introduction  of  wrong  words  also  appears  stable.  However, 
tho.  process  of  introducing  error  into  a  transcript  appears  complex,  as  indi¬ 
cated  by  the  pattern  of  correlation  between  wrong  and  right  responses — nega¬ 
tive  at  low  noise  level  and  positive  at  higher  levels — and  by  the  fact  that 
wrong  scores  proved  less  predictable  than  did  right  scores  by  means  of  tests 
of  the  Army  Classification  Battery. 

Individual  differences  were  significantly  predicted  by  several  ACB  tests 
(Verbal,  Army  Clerical  Speed,  Arithmetic  Reasoning) ,  but  development  of 
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instruments  usefully  predictive  of  performance  in  the  operational  situation 
evidently  cannot  rely  wholly  on  conventional  paper-and-pencil  tests. 


Stichman,  E.  P.,  and  Renaud,  G.  E.  Information  extraction  from  voice  com¬ 
munications  :  Work  methods  for  single  transcribers.  Technical  Research 
Note  154,  June  1965.  (AD  618  322) 

Studies  are  conducted  by  the  Combat  Communications  Task  in  voice-radio 
communications  techniques  and  overall  performance  of  personnel  involved  in 
communications  operations.  An  experiment  was  designed  to  determine  the  ef¬ 
fects  of  20  different  transcription  methods  on  the  performance  of  12  Army 
enlisted  men  in  transcribing  word  lists  received  at  4  signal-to-noise  levels. 
Three  aspects  of  communications  procedure  were  examined:  listening  to  the 
message,  writing  the  message  as  it  was  heard,  and  using  a  previous  transcript 
of  the  message  as  an  aid  in  re listening  and  retranscribing. 

Significant  improvement  in  performance  was  obtained  when  subjects  both 
listened  to  and  wrote  the  word  lists  more  than  once.  Performance  did  not 
improve  at  any  signal-to-noise  ratio  when  listening  to  a  given  word  list 
more  than  once  before  writing  the  transcript  nor  when  the  subjects  used  a 
first  or  second  transcript  as  a  reference  aid  in  retranscribing  a  word 
list.  The  conclusion  was  that  although  the  actual  absolute  gain  in  accu¬ 
racy  was  low,  there  was  sufficient  improvement  to  warrant  repeated  listening 
and  repeated  writing  of  messages  when  reception  conditions  are  below  the 
marginal  level  of  channel  communications. 


Stichman,  E.  P.  Transcriber  confidence  in  relation  to  accuracy  of  transcrip¬ 
tion.  Technical  Research  Note  175,  July  1966.  (AD  642  544) 

In  continuing  research  conducted  under  controlled  laboratory  conditions , 
the  Combat  Communications  Task  is  investigating  voice  radio-telephone  communi¬ 
cations  techniques  and  improvement  of  performance  of  personnel  in  communica¬ 
tions  operations.  The  present  study  was  designed  to  determine  the  relation¬ 
ship  of  transcriber  confidence  to  transcription  accuracy.  Measures  of  two 
aspects  of  performance — message  intelligibility  and  expressed  confidence  in 
the  correctness  of  transcription — were  obtained  from  eight  Army  enlisted  men 
(untrained  in  communications)  transcribing  word  lists  received  at  three 
signal-to-noise  ratios.  Confidence  was  expressed  through  the  use  of  a  five- 
point  rating  scale  ranging  from  "fully  confident"  to  "not  at  all  confident," 
and  a  separate  rating  was  assigned  to  each  word  in  a  list  as  it  was  transcribed. 

A  significant  relationship  was  obtained  between  transcriber  confidence 
and  accuracy  of  transcription.  As  expected,  both  mean  intelligibility  and 
mean  confidence  ratings  increased  as  a  direct  function  of  signal-to-noise 
ratio.  Signal-to-noise  ratio  did  not  similarly  affect  the  relationship  be¬ 
tween  confidence  and  accuracy.  Because  of  generally  unpredictable  and  un¬ 
stable  listening  conditions  in  the  field,  the  relationship  between  confidence 
and  accuracy — averaged  across  signal-to-noise  ratios — apears  to  provide  a 
stable  measure  which  is  the  best  practical  basis  for  estimating  transcript 
accuracy.  The  conclusion  was  that  a  positive  relationship  exists  between 
transcriber  confidence  of  correct  reception  and  message  intelligibility, 
even  when  personnel  untrained  in  communications  serve  as  transcribers.  While 
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far  from  perfect,  the  relationship  is  sufficient  to  warrant  further  research 
using  trained  communications  transcribers. 


Stichman,  E.  P.  Relationship  of  expressed  confidence  to  accuracy  of  tran¬ 
scription  by  operational  communications  personnel.  Technical  Research 

Note  192,  October  1967.  (AD  662  069) 

The  study  sought  to  determine  whether  operational  communications  per¬ 
sonnel  can  rate  their  performance  in  transcribing  voice  radio  messages  par¬ 
tially  embedded  in  noise  with  sufficient  precision  for  the  ratings  to  have 
potential  operational  utility.  Eight  experienced  communications  operators 
rated  their  confidence  in  the  accuracy  of  their  reception  and  transcription 
of  messaqes  received  at  three  signal-to-noise  ratios  (-6  dB,  0  dB,  +6  dB) , 
using  a  five-point  rating  scale.  As  a  control,  subjects  also  received  and 
transcribed  the  messages  without  making  ratings.  Measures  of  transcript 
accuracy  and  expressed  confidence  in  transcription  were  compared  with  results 
from  a  prior  study  in  which  subjects  had  no  formal  training  or  experience  in 
communications  or  transcription  (Technical  Research  Note  175) .  Experienced 
operators  were  highly  successful  in  judging  their  own  accuracy,  the  relation¬ 
ship  between  confidence  and  accuracy  being  rtet  *  .78.  Some  overconfidence 
at  the  upper  end  of  the  rating  scale  and  underconfidence  at  the  lower  end 
were  evident.  Intelligibility  improved  from  20%  to  88%  as  signal-to-noise 
ratio  increased.  The  communications  operators  performed  better  than  the 
noncommunications  trained  subjects  in  the  former  study  both  in  accuracy  of 
transcription  and  in  precision  of  confidence  ratings.  Judging  the  transcrip¬ 
tion  did  not  affect  the  average  accuracy  of  the  transcripts  in  either  study. 
In  both  studies,  subjects  tended  to  make  effective  use  of  less  than  all  five 
points  of  the  confidence  rating  scale. 


Stichman,  E.  P.  Intelligibility  differences  among  monosyllabic  words. 

Research  Memorandum  68-1,  January  1968. 

The  objective  of  this  experiment  was  to  explore  the  possibility  that 
monosyllabic  words  selected  from  a  base  of  1,000  words  and  used  to  construct 
phonetically-balanced  (PB)  word  lists  were  sufficiently  different  in  intel¬ 
ligibility  to  be  potentially  useful  in  developing  a  noise-resistant  language. 
High  and  low  intelligibility  words  were  selected  from  an  intelligibility- 
confidence  rating  scatterplot  of  PB  word  scores  from  previous  research  and 
assembled  into  high  and  low  intelligibility  lists  of  50  words  each.  Trained 
subjects  transcribed  these  at  three  signal-to-noise  ratios.  Scores  on  the 
high  intelligibility  lists  were  significantly  higher  and  scores  on  the  low 
intelligibility  lists  were  significantly  lower  in  intelligibility  than  the 
scores  on  the  standard  PB  lists. 


Castelnovo,  A.  E.  Effects  of  spectrum  sampling  on  speech  intelligibility. 

Technical  Research  Note  207 ,  March  1969. 

In  the  Monitor  Performance  Work  Unit,  ARI  studies  the  effects  of  factors 
associated  with  the  signal,  the  monitoring  task,  the  environment,  and  the 
individual  in  a  variety  of  combinations.  One  segment  of  work  unit  effort  is 
concerned  with  variables  affecting  the  intelligibility  of  audio  signals  and 
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speech,  with  present  emphasis  on  the  area  of  spectrum  selection  and  binaural 
listening.  The  current  publication  reports  on  a  study  of  the  effect  on  the 
intelligibility  of  phonetically  balanced  (PB)  words  of  excising  several  nar¬ 
row  bands  from  a  curtailed  speech  spectrum  (1,300  cycles).  A  spectrum  com¬ 
posed  of  several  discrete  pass  bands  was  compared  to  (a)  the  total  curtailed 
spectrum,  (b)  the  curtailed  spectrum  with  one  large  segment  removed  from  the 
end,  and  (c)  the  articulation  predicted  by  the  Articulation  Index.  Eighteen 
PB  word  lists  uttered  at  three  speech-to-noise  ratios  constituted  the  stimu¬ 
lus  material  presented  to  36  subjects  through  a  filter  system  with  selected 
pass  bands.  Results  indicated  that  at  the  higher  speech-to-noise  ratios, 
eliminating  several  narrow  bands  from  the  spectrum  does  not  result  in  a 
corresponding  reduction  in  intelligibility.  When  the  speech  is  35  dB  above 
the  noise,  a  reduction  of  20%  or  more  can  be  made  in  bandwidth  without  no¬ 
ticeable  reduction  in  intelligibility.  As  the  speech-to-noise  ratio  is  de¬ 
creased,  the  decrement  in  intelligibility  becomes  mere  nearly  proportional 
to  the  decrease  in  bandwidth.  A  given  bandwidth  distributed  over  a  spectrum 
area  is  more  effective  than  an  equivalent  bandwidth  massed  in  one  part  of 
the  spectrum.  Distributed  sampling  of  the  spectrum  was  found  to  be  more  ef¬ 
fective  than  would  be  expected  from  Articulation  Index  computations. 


House,  J.  L.,  and  Cohen,  S.  L.  Current  procedures  in  selection  training  and 
utilization  of  voice  processors  (U) .  Research  Study  70-4,  April  1970. 
(CONFIDENTIAL) 

Opinions  of  a  cross  section  of  linguistic  personnel  concerning  present 
procedures  and  suggestions  for  changes  in  the  selection,  training,  assign¬ 
ment,  and  career  development  of  the  Army  transcriber  are  discussed. 


Dean,  R.  F.  Operator  performance  variables  in  LeFox  Grey  (U) .  Research 

Study  70-5,  May  1970.  (CONFIDENTIAL) 

The  complex  weapons  and  surveillance  systems  of  the  modern  Army  have 
created  a  relatively  new  series  of  jobs  requiring  operators  to  monitor  in¬ 
strument  panels,  radar  scopes,  communications  nets,  and  other  types  of  sig¬ 
nal  detection  apparatus. 

The  present  research  study  reports  on  observations  made  by  ARI  scien¬ 
tists  of  a  computer-assisted  communications  processing  system  with  a  view 
to  identifying  variables  which  adversely  affect  human  operator  performance. 


House,  J.  L.,  and  Cohen,  S.  L.  Quantitative  evaluation  of  current  procedures 
in  voice  processing  (U) .  Technical  Research  Report  1174,  July  1971. 
(CONFIDENTIAL)  (AD  516  989) 

In  the  conduct  of  human  performance  experimentation  research,  ARI  has  a 
primary  objective  to  improve  performance  in  Army  monitor  jobs,  with  special 
emphasis  on  developing  and  testing  new  work  methods.  An  important  segment 
of  the  research  is  concerned  with  human  factors  studies  in  communication 
analysis  and  processing,  the  present  report  describes  a  study  primarily 
directed  to  evaluating  methods  of  gisting,  that  is,  abstracting  essential 
elements  of  a  voice  communication  as  a  means  of  identifying  problems  in  ex¬ 
tracting  intelligence  information  from  voice  communications. 
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House,  J.  L.  A  comparison  of  tw'o  methods  of  gisting.  Technical  Research 

Note  236,  March  1972.  (AD  743  185) 

An  important  segment  of  ARI  research  on  monitor  performance  is  devoted 
to  human  factors  studies  in  communication  analysis  and  processing.  The 
present  publication  reports  on  research  to  evaluate  two  experimental  methods 
of  gisting— that  is,  abstracting  the  essential  elements  of  a  voice  message- 
in  Comparison  with  baseline  performance  in  simulated  on-line  gisting. 

Twenty-four  communications  processors  were  each  tested  using  two  methods 
of  gisting.  In  one  method  (free  repeat) ,  the  processor  listened  once  to  the 
complete  message  with  no  option  to  atop  or  replay,  preparing  as  complete  a 
gist  as  he  could,  and  was  then  free  to  replay  the  tape  as  he  judged  necessary 
to  complete  the  gist.  In  the  second  method  (forced  repeat) ,  the  processor 
listened  to  the  complete  message  once  and  prepared  as  complete  a  gist  as  he 
could.  He  then  listened  to  the  complete  message  five  additional  times  with 
no  option  to  stop- or  replay. 

The  free  repeat  method  produced  a  higher  quality  gist  and  a  higher  per¬ 
centage  of  key  items  correctly  reported  them  did  the  forced  repeat  method. 

The  free  repeat  method  also  took  lesS  time  to  produce  o  complete  gist.  How¬ 
ever,  the  time  spent  on  each  repetition  was  controlled  in  the  forced  repeat, 
method.  In  both  methods,  repetition  produced  gains  in  number  of  key  items 
reported  correctly  and  in  the  overall  quality  of  the  gist. 


Cohen,  S.  L,,  and  Turney,  J,  R.  Comparison  of  operator  gisting  skills  in 
LeFox  Grey  and  standard  systems  (U) .  Technical  Research  Report  1184, 
November  1974.  (CONFIDENTIAL)  (AD  C000  382) 

An  experimental  semiautomated  communication  processing  system,  LeFox 
Grey,  was  tested  in  parallel  field  operation  with  the  current  standard  sys¬ 
tem.  Comparisons  were  made  to  determine  whether  differences  in  the  output 
of  the  two  systems  were  attributable  to  system  differences  or  to  differences 
in  operator  proficiency. 

The  research  showed  that  operators  in  the  LeFox  Grey  system  and  in  the 
standard  system  did  not  differ  in  gisting  ability.  Operators  in  both  groups 
produced  better  gists  from  the  message  containing  more  items  of  critical 
information — better  in  both  percentage  of  correct  identifications  and  in 
rated  quality  of  gists.  Therefore,  results  of  comparisons  between  the  LeFox 
Grey  system  and  the  standard  system  were  functions  of  aspects  of  the  man- 
machine  interface  of  the  systems  and  not  of  differences  in  operator 
proficiency. 


Cohen,  S.  L.  Comparative  work  environment  perceptions  of  operating  per¬ 
sonnel  within  experimental  and  standard  communication  systems.  Re¬ 
search  Report  1180,  February  1975.  (AD  A005  215) 

An  experimental  semiautomated  field  communication  system  was  evaluated 
in  comparison  with  the  current  standard  communication  system.  Scanners  in 
the  experimental  system  and  operators  and  transcribers  on  the  standard  sys¬ 
tem  completed  two  questionnaires  dealing  with  past  training  and  experience 
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and  perceptions  of  jobs  in  the  system.  Responses  were  analysed  with  special 
reference  to  acceptability  of  the  equipment  and  working  aids,  adequacy  of  job 
assistance  and  performance  feedback,  superior /subordinate  and  peer/peer  in¬ 
teraction,  and  the  individual's  perception  of  his  role  in  the  system.  Re¬ 
sults  showed  that  the  system  being  evaluated  was  superior  to  the  standard 
system.  Neither  system  provided  adequate  procedures  for  man-man  interaction. 
Insufficient  feedback  on  performance,  and  little  positive  recognition,  guid¬ 
ance,  or  job  assistance  were  provided  for  operating  personnel.  Work  aids 
were  generally  unsatisfactory. 

Personnel  were  able  to  perform  for  longer  periods  of  time  with  less 
noticoable  fatigue  in  the  experimental  system  than  in  the  standard  system. 

The  research  has  provided  leads  for  improving  motivation  through  improved 
utilization  of  feedback  information  and  the  evaluation  of  such  feedback  in 
terms  of  system  productivity. 


Shields,  J.  L.  Speed  listening  for  message  classification.  Technical  Re¬ 
search  Memorandum  75-2,  March  1975.  (FOR  OFFICIAL  USE  ONLY) 

Listeners  attempted  to  identify  the  subject  matter  of  tapes  with  four 
types  of  communication.  Rate  of  presentation  had  a  significant  effect.  Best 
performance  occurred  at  regular  speed  followed  by  1.5  times,  then  2  times. 
This  general  pattern  held  for  most  but  not  all  communication  types.  In  gen¬ 
eral,  participants  were  unable  to  accurately  state  the  subject  matter  of  the 
communication  at  2  times;  they  were  able  to  state  a  part  of  the  communication 
at  1.5  times.  There  was  a  close  agreement  between  the  participant's  subjec¬ 
tive  estimates  of  their  performance  and  their  ability  to  identify  the  subject 
matter  of  communications  presented.  The  report  suggested  directions  for 
further  study. 


Cohen,  S.  L. ,  and  Turney,  J.  R.  Results  of  an  organizational  diagnostic 
survey  of  an  Army  field  facility  work  environment.  Technical  Paper 
272,  January  1976,  (AD  A020  934) 

The  aim  of  organizational  effectiveness  research  is  to  increase  human 
performance  effectiveness  in  an  organization  and  to  improve  teamwork  and 
job  satisfaction,  by  developing  diagnostic  instruments  to  identify  problem 
areas,  intervening  with  organizational  development  techniques  to  correct  the 
problems,  and  finally  evaluating  the  intervention  results  in  terms  of  pro¬ 
ductivity  and  job  satisfaction. 

A  diagnostic  Work  Environment  Questionnaire  (WEQ)  was  developed  and 
validated  over  a  3-year  period  at  an  Army  field  installation.  It  elicited 
from  both  supervisors  and  subordinates  their  attitudes  and  perceptions  on 
their  job  duties,  training,  performance  standards,  and  consequences,  and 
on  their  organizational  supervision,  work  group,  job  importance,  and  feed¬ 
back.  Three  versions  were  tailored  with  job-specific  items  for  a  super¬ 
visory  NCO  position  anJ  two  different  subordinate  positions;  the  items  are 
readily  adapted  to  a  variety  of  actual  duties.  Data  from  the  1972  pretest 
and  1973  survey  supplemented  the  1974  WEQ  survey  reported  in  detail  in  this 
report . 
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Seven  major  organizational  problem  areas  were  identified  in  both  1973 
and  1974 i  peer  group  norms  which  failed  to  encourage  good  performance,  in¬ 
sufficient  performance  feedback  need  for  training  in  supervisory  techniques, 
role  ambiguity  and  conflict,  inadequate  intergroup  communication  patterns, 
lack  of  clear  performance- reward  relationsnips ,  and  smoiguous  performance 
evaluation  standards. 

A  program  of  active  intervention  has  since  been  designed  and  implemented 
to  reduce  these  problems;  a  resurvey  has  indicated  that  the  intervention  did 
successfully  decrease  certain  problems  and  increase  job  satisfaction  and 
performance . 


Turney,  J.  P.. ,  and  Cohen,  S.  L.  The  development  of  a  work  environment  ques¬ 
tionnaire  for  the  identification  of  organizational  problem  areas  in 
specific  Army  work  settings.  Technical  Paper  275,  June  1976. 

(AD  A028  241) 

The  aim  of  organizational  effectiveness  research  is  to  increase  human 
performance  effectiveness  in  an  organization  and  to  improve  teamwork  and  job 
satisfaction,  by  developing  diagnostic  instruments  to  identify  problem  areas, 
intervening  with  organizational  development  techniques  to  correct  the  prob¬ 
lems,  and  finally  evaluating  the  intervention  results  in  terms  of  productivity 
and  job  satisfaction.  This  report  discusses  the  development  and  validation 
of  the  Work  Environment  Questionnaire  (WEQ)  which  is  used  to  identify  organi¬ 
zational  problem  areas  and  evaluate  interventions. 

The  WEQ  was  developed  and  validated  over  a  3-year  period  as  part  of  an 
organizational  effectiveness  program  at  an  Army  field  installation.  Focusing 
on  individual  work  motivation  and  the  situational  factors  affecting  this 
motivation,  it  is  designed  to  elicit  from  both  supervisors  and  subordinates 
their  attitudes  and  perceptions  on  their  job  duties,  training,  performance 
standards,  and  consequences,  and  on  their  organizational  supervision,  work 
group,  job  importance,  and  feedback.  Section  formats  are  designed  to  apply 
to  a  wide  range  of  Army  settings,  but  item  content  is  specific  to  the  Army 
work  setting  and  job.  Three  questionnaires  were  tailored  with  job-specific 
items  for  a  supervisory  NCO  position  and  two  different  subordinate  positions; 
the  three  as  a  whole  comprise  the  WEQ.  Validation  procedures  significantly 
correlated  the  attitude  measures  with  independent  measures  of  performance 
and  ef  ort. 

The  WEQ  provides  indexes  of  soldier  perceptions,  motivations,  and  satis¬ 
factions  in  specific  terms  which  can  identify  problem  areas  suitable  for 
corrective  intervention.  A  further  survey  in  the  field  installation  has 
since  identified  seven  major  problem  areas,  and  a  program  of  active  inter¬ 
vention  has  been  designed  to  reduce  these  problems. 
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deHaan,  H.  J.  A  speech-rate  intelligibility/comprehensibility  threshold  for 

speeded  and  time-compressed  connected  speech.  Technical  Paper  297, 

June  1978.  (AD  A061  172) 

(Also  in  Perception  and  Psychophysics,  1977,  22_,  366-372.) 

The  speech  rapidity  threshold  is  a  new  method  for  determining  the  maxi¬ 
mum  rate  of  speech  understood  by  individual  listeners.  It  involves  repeated 
judgments  of  the  comprehensibility  of  speech  as  the  rate  of  speech  varies. 
The  purpose  of  the  present  experiment  was  to  determine  the  relationship  of 
the  speech  rapidity  threshold  to  the  intelligibility  and  comprehension  of 
speech . 

To  accomplish  this,  two  experiments  were  conducted.  The  first  experi¬ 
ment  compared  two  types  of  speech  reportedly  different  in  intelligibility, 
simple  speeded  speech  and  compressed  speech.  Thresholds  for  compressed 
speech  were  significantly  higher  than  those  for  speeded  speech,  an  indica¬ 
tion  that  the  threshold  at  the  very  least  reflects  intelligibility. 

The  second  experiment  correlated  threshold  values  with  traditional  com¬ 
prehension  test  scores.  In  general,  correlations  were  low,  an  indication 
that  the  method  is  not  a  measure  of  comprehension.  Results  were  interpreted 
to  mean  that  the  threshold  reflects  an  intermediate  level  of  information 
processing  involving  the  perception  of  the  potential  for  interpretation  or 
comprehension  rather  than  the  complete  act  of  comprehension  per  se. 


deHaan,  H.  J. ,  and  Schjelderup,  J.  R.  Threshold  of  intelligibility/compre¬ 
hensibility  of  rapid  connected  speech;  Method  and  instrumentation. 

Technical  Paper  296,  June  1978.  (AD  A060  556) 

(Also  in  Behavior  Research  Methods  S  Instrumentation,  1978,  bO,  841-844.) 

The  technology  of  speech  compression  demands  new  methods  for  measuring 
the  intelligibility  and  comprehensibility  of  compressed  speech.  The  present 
report  describes  a  method  in  which  repeated  judgments  of  the  comprehensibility 
of  speech  are  obtained  as  the  rate  of  speech  is  made  to  vary.  The  method  is 
similar  to  the  B4k6sy  method  for  the  determination  of  pure-tone  thresholds 
but  differs  from  it  in  that  rate  of  speech  is  varied  rather  than  the  inten¬ 
sity  of  an  auditory  signal.  The  result  is  a  threshold  value  representing 
the  maximum  rate  of  speech  judged  comprehensible  by  an  individual  listener. 
Instrumentation  that  varies  speech  rate,  accelerating  or  decelerating- the 
rate  either  with  or  without  pitch  changes,  was  developed  for  this  determi¬ 
nation  and  is  described  in  some  detail.  Both  the  method  and  associated  in¬ 
strumentation  have  potential  for  a  variety  of  applications . 


Lambert,  J.  V.,  Shields,  J.  L. ,  Gade,  P.  A.,  and  Dressel,  J.  D.  Comprehen¬ 
sion  of  time-compressed  speech  as  a  function  of  training.  Technical 
Paper  295,  June  1978.  (AD  A058  204) 

This  experiment  studied  five  training  methods  for  effective  listening 
of  time-compressed  speech.  Army  subjects  trained  by  two  methods  using  in¬ 
centives  understood  speech  played  2.2  times  faster  than  normal  without  de¬ 
gradation  of  performance.  Subjects  trained  by  three  methods  not  using 
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incentives  had  significantly  degraded  performance  when  compressed  speech  was 
presented  at  rates  faster  than  1.85  times  the  normal  rate. 


Gade,  P.  A.,  and  Gertman,  D.  Listening  to  compressed  speech:  The  effects 

of  instructions,  experience,  and  preference.  Technical  Paper  369, 

August  1979.  (AD  A075  408) 

This  experiment  attempted  to  assess  the  effects  of  different  information¬ 
seeking  strategies  on  the  rate  at  which  individuals  chose  to  listen  to  pas¬ 
sages  of  time-compressed  speech,  and  on  their  comprehension  of  those  passages. 
The  research  also  assessed  the  effects  of  prior  experience  with  compressed 
speech  on  listening  rates  and  on  comprehension  of  48  Army  enlisted  men  and 
women.  The  various  information-seeking  strategies  employed  produced  no  sig¬ 
nificant  differences  in  speed  or  accuracy  of  performance.  Personnel  preferred 
to  listen  to  speech  rates  well  above  normal  speaking  rates  (j>  <  .001) .  Al¬ 
though  prior  experience  with  compressed  speech  did  not  influence  preferred 
listening  rates,  it  did  influence  the  participants'  listening  rates  when  they 
were  induced  to  listen  to  speech  as  rapidly  as  possible  (j>  <  0.001) .  Results 
are  discussed  in  terms  of  Berlyne's  epistemic  curiosity  hypothesis  and  in  re¬ 
lation  to  the  results  of  preference  research  by  Lass,  Foulke,  Nester,  and 
Comerci  (1974) . 


Martinek,  H.,  Johnson,  E.  M.,  and  Jeffrey,  T.  Communications  intelligence: 

Transcriber  output  enhancement.  Research  Report  1268,  September  1980. 

A  workflow  model  of  major  transcriber  functions  was  developed  and  time 
data  for  these  functions  were  collected  on  52  operational  personnel.  A  time 
analysis  was  performed  with  respect  to  individual  functions  and  logical  sets 
of  functions  for  a  representative  sample  for  the  operationally  meaningful 
groups  of  transcribers .  The  time  analysis  of  transcriber  functions  indicated 
that  about  one-third  of  productive  time  was  spent  in  the  Listen,  Rock,  and 
Search  functions  and  another  one-third  in  doing  these  functions  in  conjunc¬ 
tion  with  others.  About  11%  of  the  transcribers'  time  was  spent  in  obtaining 
hexu  from  reference  material,  5%  in  the  tape  changing  functions,  and  8%  in 
team  work  functions.  Differences  clearly  existed  between  organizational 
groups  in  the  time  spent  on  specific  transcriber  functions.  Development  of 
procedures  to  enable  transcribers  to  exploit  digitization  and  computer  stor¬ 
age  of  taped  communications  should  improve  productivity  at  least  10%  to  20% 
by  eliminating  search  and  tape  changing  functions,  by  reducing  time  spent  in 
getting  help  from  reference  material  and  other  transcribers,  and  in  giving 
help,  and  improving  exploitation  of  source  material. 
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Simulation  and  Gaming:  General 
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Tiede,  R.  V.,  Burt,  R.  A.,  and  Bern,  T.  T.  Design  of  an  integrated  division- 

level  battle  simulation  for  research,  development,  and  training.  Vol¬ 
ume  I.  Technical  Report  420,  August  1979.  (AD  A082  761) 

The  requirements  of  this  study  were:  (a)  to  develop  a  top-down  design 
for  an  integrated  family  of  modular  division- level  battle  simulations  which 
separately  or  jointly  will  exercise  players  performing  critical  functions 
in  command  and  control  (Vol.  I) ?  and  (b)  to  develop  detailed  design  specifi¬ 
cations  for  the  Intelligence  Staff  module  of  the  integrated  battle  simula¬ 
tion  (Vol.  II). 

The  design  approach  involved  the  following  principles:  selection  of 
the  decision  variables  to  be  manipulated  by  the  division  staff  modules  and 
the  event  threads  both  within  and  external  to  the  staff  modules  thereby 
fixing  the  event  sequence  and  time  of  occurrence?  and  incorporating  into 
the  simulation  every  event  thread  needed  to  support  the  input/output  rela¬ 
tionship.  The  dynamic  realism  needed  to  place  decisionmakers  in  a  realistic 
environment  is  achieved  by  means  of  an  event  store  technique.  The  five 
classes  of  events  of  which  the  event  threads  are  composed  are  defined  and 
the  logical  flow  of  the  event  store  simulation  is  illustrated.  A  sixth 
class  cf  • jnt  needed  for  operation  in  an  ADP-assisted  mode  is  also  defined. 
The  appr  a ;n  begins  at  the  heart  of  the  information  system,  t'-ie  decisions, 
and  then  develops  the  simulation  needed  to  implement  them — the  inverse  of 
the  usual  approach. 

The  design  concept  provides  for  a  man  in  the  loop  in  that  any  one  or 
any  combination  of  five  basic  staff  modules  (Cmd  Grp,  G2,  G3,  G1/G4,  FSCE) 
plus  one  enemy  module  (Cmd  Grp)  may  be  either  occupied  by  human  players  or 
simulated.  The  module  simulations  are  designed  as  "plug-in"  modules,  any 
one  or  more  of  which  can  be  replaced  by  players.  The  simulation  also  con¬ 
tains  a  battle  outcome  generator  which  simulates  all  other  events  within 
the  division  and  the  enemy  force  opposing  it,  and  which  feeds  back  to  the 
players  in  slow,  real,  or  fast  time  (at  the  option  of  the  user)  the  results 
of  their  decisions.  The  design  also  provides  for  interfaces  with  higher 
and  adjacent  units.  It  includes  the  following  features: 

1.  The  modules  are  based  on  the  traditional  G-staff  structure. 

2.  Nuclear  battle  events  are  included. 

3.  Live  modules  may  be  required  to  perform  simultaneous  planning  and 
execution?  the  results  of  such  planning  may  be  evaluated  by  subse¬ 
quent  execution  of  plans . 

4.  Other  staff  elements  not  included  in  the  basic  five  modules  (e.g., 
engineer,  signal)  are  "hardwired"  components  of  the  simulation. 

5.  The  basic  design  provides  for  manual  operation  by  live  players  but 
it  is  readily  expandable  to  permit  player  operation  in  an  ADP- 
assisted  mode. 


V-l 


*\ 


Simulation  and  Gaming*.  General 


Tiede,  R.  V. ,  Burt,  R.  A.,  and  Bern,  T.  T.  On  the  design  of  simulations  of 
command  and  control  processes.  Research  Note  80-39,  in  press  (Novem¬ 
ber  1980). 

This  report  covers  two  additional  design  requirements  to  an  exiting 
design  for  a  battle  simulation  intended  for  research,  development,  or  train¬ 
ing.  The  first  requirement  is  to  simulate  the  command  control  processes 
so  that  they  reflect  human  performance  in  the  information  processing  and 
decision-making  sequences.  The  second  is  to  provide  alternative  configura¬ 
tions  for  the  "player"  staff  modules  so  that  the  investigator  may  select  the 
configuration  best  suited  to  his  behavioral  research  objectives.  Both  prob¬ 
lems  are  approached  by  combining  the  known  human  factors  in  command  and  con¬ 
trol  with  a  detailed  structuring  of  the  action  processes  employed  by  the 
command  group/staff.  All  relevant  aspects  of  human  performance  except  the 
higher-level  decision-making  operations  can  be  realistically  simulated.  A 
trade-off  is  found  to  exist  in  the  alternative  configurations  between  the 
realism  of  staff  play,  the  ability  to  make  behavioral  measurements,  and  the 
cost  of  personnel  and  computer  hardware/software. 


Tiede,  R.  V. ,  Burt,  R.  A.,  and  Bern,  T.  T.  Some  guidelines  for  effective 
task  design  in  command  and  control  simulations.  Research  Note  80-40, 
in  press  (November  1980) . 

This  report  develops  principles  of  task  design  for  interactive  simula¬ 
tions  in  which  staff  players  execute  the  command  and  control  of  a  simulated 
battle.  A  companion  report  on  the  design  of  simulations  of  command  and 
control  processes  presents  the  basic  design  framework  of  a  battle  simulation 
capable  of  application  alternatively  to  research,  development,  or  training. 
This  framework  includes  consideration  of  alternative  configurations  of  the 
"player"  staff  modules  so  that  an  investigator  may  select  a  configuration 
best  suited  to  his  behavioral  research  objectives.  The  guidelines  for  ef¬ 
fective  task  design  are  developed  from  the  alternative  configurations. 
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Ryan,  T.  G.  Studies  of  tactical  military  decision  making  II.  An  informa¬ 
tion  network  aid  to  scenario  development.  Research  Study  69-11,  Sep¬ 
tember  1969. 

Current  military  field  manuals  and  military  information  system  reports 
were  used  to  develop  a  Tactical  Decision  Information  Network  to  aid  in  the 
preparation  of  test  scenarios  to  support  research  concerned  with  the  flow  of 
tactical  information  and  its  relationship  to  decision-making  behavior  in 
the  Army's  tactical  operations  systems.  From  these  official  sources,  command 
unit  organization,  command  decision-making  responsibilities,  and  information 
requirements  of  staff  elements  were  defined.  Within  the  sequence  of  actions 
in  developing  and  executing  a  course  of  action,  decision  points  were  defined, 
decisionmakers  were  specified,  and  information  categories  were  delineated. 
These  data  were  then  organized  in  the  form  of  function  flow  block  diagrams 
and  associated  data  input  sheets  which,  collectively,  constitute  the  Network. 

The  Network  provides  a  framework  of  checklists  for  information  inputs 
and  flow  diagrams  encompassing  the  total  information  analysis  and  decision¬ 
making  sequence.  Sections  of  the  Network  can  be  delimited  according  to  re¬ 
quirements  of  the  problem  to  be  studied  experimentally. 


Krumm,  R.  L. ,  Rowe,  C.  H.,  and  Torpey,  F.  E.  Research  on  tactical  military 
decision  makings  I.  Design  of  a  Simulated  Tactical  Operations  System 
(SIMTOS).  Research  Problem  Review  70-1,  October  1970. 

Conditions  necessary  for  a  realistic  test  facility  environment,  scenario 
form  and  content,  and  testing  practicalities  were  delineated.  On  the  basis 
of  these,  plans  were  outlined  in  brief  for  a:  sequence  of  research  designed 
to  provide  a  capability  to  study  decision-making  behavior  during  free-play 
two-sided  war  games  using  a  simulated  tactical  operations  system  (SIMTOS) 
for  laboratory  research. 

SIMTOS  capabilities  ,ilable  in  ARI  to  support  the  research  were  ex¬ 
amined  and  additional  support  capabilities  needed  were  identified.  Details 
of  the  required  physical  facility  are  given.  The  report  provides  the  broad 
outlines  of  a  six-phase  research  plan  designed  to  produce  a  simulated  tacti¬ 
cal  operations  system  (SIMTOS)  for  exercise  within  a  simulated  tactical 
operations  center  (SIMTOC) . 


Krumm,  R.  L. ,  Robins,  J.  E.,  and  Ryan,  T.  G.  Research  on  tactical  military 
decision  making:  III.  Predictor  variables  and  criterion  measures. 
Technical  Research  Note  229,  May  1973.  (AD  765  457) 

The  Command  Systems  program  is  concerned  with  human  factors  problems  of 
information  presentation,  processing,  and  utilization  in  command  and  control 
systems.  One  major  objective  is  to  provide  research  findings  by  which  infor¬ 
mation  assimilation  and  decisionmaking  may  be  facilitated.  The  present  pub¬ 
lication  describes  research  to  develop  and  evaluate  a  scenario  for  a  test  of 
tactical  military  decisionmaking  and  to  derive  methods  for  scoring  the 
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decision-making  behavior  for  use  in  a  broad  program  of  manned  systems  re¬ 
search  to  improve  tactical  decisionmaking. 

The  experimentation  was  conducted  by  personnel  of  the  BUNKER-RAMO 
Corporation  in  the  Simulated  Tactical  Operations  System  (SIMTOS)  facility. 

A  test  scenario  was  developed  and  administered  individually  to  20  senior 
field  grade  officers.  The  assigned  task  required  each  officer  to  wrive  a 
defense  plan  for  his  division  sector  against  an  expected  attack  by  two 
mechanized  infantry  divisions .  The  scenario  was  presented  by  using  cathode 
ray  tube  (CRT)  displays ,  computer -driven  typewriters ,  and  random  access 
slide  projection  equipment.  Defense  plans  were  scored  using  USA  CGSC  school 
solutions  as  criteria.  Two  scoring  procedures  were  utilized.*  (a)  Leaven¬ 
worth  Standard  (based  on  rationales  and  solutions  in  the  CGUC  lesson  plans) 
and  (b)  Consensus  Standard  (to  provide  for  computing  average  subject  re¬ 
sponses  in  the  event  CGSC  Standards  were  inappropriate  as  a  result  of  sce¬ 
nario  changes) . 

The  experiment  demonstrated  the  practicality  of  developing  a  priori 
scoring  standards  for  complex  decision-making  tasks.  The  test  proved  re¬ 
liable.  Measures  of  the  decision-making  behavior  of  the  officers  were  highly 
correlated  with  the  criterion  score.  A  combination  of  four  measures,  Ex¬ 
perience,  Ability,  Decision  Process  Pattern,  and  Significant  facts,  were 
highly  effective  as  criterion  score  predictors. 


Robins,  J.  E.,  Buffardi,  L. ,  and  Ryan,  T.  G.  Research  on  tactical  military 

decision  making:  Applications  of  a  decision  prediction  concept  in  a 

SIMTOS  environment.  Technical  Paper  246,  March  1974.  (AD  780  812) 

The  present  experiment  was  conducted  to  determine  whether  predictors  of 
decision  qua.’ity  established  in  a  prior  experiment — based  on  academic  achieve¬ 
ment  and  information  processing  strategy — maintained  their  effectiveness  in 
a  defensive  planning  scenario  incorporating  a  fully  computerized  information 
retrieval  capability.  A  test  scenario  for  defensive  planning  was  developed 
and  administered  individually  to  20  senior  field  grade  officers,  4  at  a  time. 
Access  to  data  base  and  presentation  of  stimulus  material  were  fully  auto¬ 
mated.  Each  officer,  assuming  the  role  of  a  G3  operations  officer,  planned 
a  division  defense,  in  sector,  aqainst  an  expected  attack  by  two  mechanized 
infantry  aggressor  divisions.  Decision  quality  was  scored  according  to 
standards  developed  by  the  USCGSC  at  Fort  Leavenworth.  Nine  predictors  se¬ 
lected  in  a  prior  experiment  were  included  in  the  present  analysis.  These 
were  based  on  an  individual's  career  experience,  his  academic  records  in 
staff  college,  the  information  processing  and  assimilation  strategy  he  uses, 
and  number  of  relevant  lacts  obtained  in  processing  tactical  information. 

Four  predictors  maintained  substantial  predictive  stability  in  the 
present  SIMTOS  environment  (yielding  a  multiple  correlation  coefficient 
of  .79  which  reduced  to  .59  where  corrected  for  shrinkage):  (a)  recency 
at  CGSC?  (b)  CGSC  class  standing;  (c)  information  request  slope,  the  num¬ 
ber  of  information  requests  made  by  the  officer  early  in  the  decision¬ 
making  process  in  relation  to  decision  quality;  and  (d)  terminal  pause,  the 
time  between  final  information  request  and  decision.  Predictors  demon¬ 
strated  their  effectiveness  in  a  static  tactical  planning  situation.  Final 
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determination  of  the  utility  of  the  predictor  concept  awaits  its  applica¬ 
tion  in  a  simulated  combat  environment. 


Baker,  J.  D.  SIMTOS:  A  man- in- the- loop  interactive  simulation  of  a  tacti¬ 
cal  operations  system.  In  Huber,  R.  K. ,  Jones,  L.  F.,  and  Reine,  E. 

Military  Strategy  and  Tactics.  New  York:  Plenum,  1975. 

Shubik  and  Brewer  [1]  recently  reported  findings  from  a  detailed  survey 
of  132  Models,  Simulations,  and  Games  (MSG)  used  in  the  U.S.  Department  of 
Defense  (DOD) .  Among  the  findings,  they  discovered  that  only  10%  of  the 
MSG's  dealt  with  man-machine  or  manual  system  problems.  Additionally,  they 
found  that  approximately  75%  of  the  MSG's  employed  in  the  DOD  are  large 
and  deal  primarily  with  machine  simulations  or  force  structure,  i.e.,  they 
are  mainly  machine  (or  weapons  system)  evaluation  tools.  They  also  found 
that  most  of  the  MSG's  being  used  were  not  very  scientific  and,  consequently, 
rarely  were  the  MSG's  used  as  research  tools. 

In  contrast  to  the  trends  cited  by  Shubik  and  Brewer,  the  present  paper 
will  deal  with  a  "minority"  MSG,  namely,  a  man-machine  simulation  facility. 
The  facility  under  discussion  will  be  the  SIMTOS  (Simulated  Tactical  Opera¬ 
tions  System).  It  is  a  further  oddity  among  MSG's  in  that  it  is  a  research 
tool  which  has  been  successfully  employed  to  integrate  system  users  into 
system  design  procedures.  What  follows  is  a  delineation  of  the  scope  and 
limitations  of  SIMTOS,  a  brief  description  of  the  physical  facility  employed, 
a  summary  of  the  research  and  findings  to  date  obtained  while  using  this 
facility,  and  a  discussion  of  some  of  the  ongoing  SIMTOS  efforts. 


Cook,  J.  R. ,  and  Herzer,  I.  Development  of  scenario  material  to  support 

two-person  play  within  SIMTOS  (I:  SIMTOS  23?  II:  SIMTOS-OD) .  Tech¬ 
nical  Report  TR-79-A1,  January  1979.  (AD  A066  781) 

The  proposed  modifications  of  the  current  SIMTOS  were  to  accommodate: 

(a)  joint  play  wherein  an  Intelligence  Officer  (G2)  and  an  Operating  Officer 
(G3)  would  interact  in  decisionmaking  and  problem  solving  in  a  simulated 
tactical  operations  center  environment  (SIMTOS-23) ;  and  (b)  to  put  a  live 
G3  in  a  defensive  (D)  role  against  a  live  G3  in  an  offensive  (O)  role 
(SIMTOS-OD).  Past  experiments  had  employed  a  computer  as  the  player’s 
opponent . 

In  SIMTOS-23,  individual  and  joint  scoring  posed  unique  problems. 

Within  a  real  world  environment  where  constant,  rapid,  and  demanding  inter¬ 
action  is  the  norm,  it  is  difficult  to  measure  with  precision  the  individual 
decision  processes  and  use  of  decision  aids.  To  achieve  the  basic  purposes 
of  a  G2/G3  SIMTOS  experiment,  certain  artificialities  were  introduced  but 
these  were  minimized. 

In  OD  SIMTOS  certain  significant  problems  were  readily  apparent.  One 
was  to  overcome  the  difficulties  an  Aggressor  G3  would  encounter  when  faced 
with  structuring  and  maneuvering  a  large  foreign  military  force  against  a 
Defender  G3  who  was  trained  for  and  had  extensive  experience  in  U.S.  force 
structure  and  maneuvering.  Secondly,  the  Aygressor  G3  had  to  function  as  a 
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live ,  independently  thinking,  and  active  individual,  not  as  a  computer.  And 
lastly,  the  quantity  and  quality  of  intelligence  to  be  provided  opposing 
G3's  had  to  be  addressed. 

This  report  is  primarily  addressed  to  military  researchers  engaged  in 
man-in-the-loop  simulations  which  investigate  player  behavior  in  coopera¬ 
tive  and  antagonistic  circumstances. 


Granda,  T.  M. ,  Johnson,  E.  M.,  Halpin,  S.  M. ,  and  Burgess,  G.  SIMTOS 

(1967-1977) :  Review  of  an  approach  to  the  experimental  analysis  of 

tactical  information  processing.  Research  Report  1293,  in  press 

(November  1980) . 

This  report  summarizes  and  evaluates  the  SIMTOS  research  program  (1967- 
1977)  and  makes  recommendations  for  future  SIMTOS  research  and/or  future 
battle  simulations. 

The  origin  and  development  of  SIMTOS  as  a  research  vehicle  is  traced, 
as  well  as  as  a  physical  simulation.  Experiments  utilizing  the  SIMTOS  are 
reviewed  and  critiqued.  Problems  encountered,  lessons  learned  from  the 
SIMTOS  are  enumerated  and  discussed.  Specific  recommendations  for  future 
tactical/battle  simulations,  in  SIMTOS  and  non-SIMTOS  environments,  are 
discussed. 

The  findings  reveal  that  SIMTOS  achieved  a  high  degree  of  face  validity 
and  it  was  considered  to  be  a  reasonable  first  approach  to  a  computerized  TOC 
research  vehicle. 

It  demonstrated  that  computer  technology  can  be  brought  to  bear  on  the 
immense  information  processing  to  tasks  inherent  in  a  division  TOC. 

However,  major  problems  hindered  the  SIMTOS  effort.  Among  the  problems 
were  many  hours  of  computer  down-time,  an  out-of-date  computer  system, 
player  acquisition,  lack  of  appropriate  tasking,  and  lack  of  consistent 
scoring  criteria.  These  problems  and  others  contributed  heavily  to  the 
fact  that  the  research  results  did  not  produce  major  behavioral  findings. 

It  was  concluded  that  a  significant  behavioral  impact  in  the  area  of 
tactical,  division  level  man-computer  information  processing  could  be  forth¬ 
coming  with  the  following  improvements  to  SIMTOS  or  SIMTOS-like  systems; 

(a)  acquisition  of  modern  reliable  and  sophisticated  computer  and  peripherals 
(e.g.,  input/output  devices);  (b)  closer  cooperation  between  military  research 
team  and  participant  sources;  (c)  careful  design  of  scenario  and  tasking  com¬ 
ponents;  and  (d)  validated  and  standardized  scoring  procedures. 

Tactical  games  and  simulations  are  being  widely  studied  by  the  U.S. 

Amy  agencies.  These  games  range  in  sophistication  and  automation.  They 
all,  however,  are  linked  by  the  fact  that  none  have  been  validated  and  none 
can  be  used  as  research  vehicles.  This  report  on  SIMTOS  can  be  used  by 
development  agencies  as  a  guideline  or  standard  in  their  current  efforts  to 
study  tactical  decisionmaking  and  information  processing  via  simulation. 
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Tremble,  T.  R. ,  and  Costner,  R.  s.  Information  flow  in  training  exercises 
with  the  Combined  Arms  Tactical  Training  Simulator  (CATTS) .  Research 
Memorandum  77-20,  November  1977. 

This  document  reports  n  study  conducted  by  the  Fort  Benning  Field  Unit 
of  ARI  on  the  demonstration  model  of  the  Combined  Arms  Tactical  Training 
Simulator  (CATTS).  The  study  was  undertaken  as  part  of  ARI's  research  sup¬ 
port  of  the  CATTS  directorate  during  system  definition  research  on  CATTS. 

The  overall  purpose  of  system  definition  research  was  to  develop  a  training 
device  requirement  and  user  input  to  technical  specifications  for  a  follow- 
on  version  of  CATTS.  The  present  study  was  to  describe  the  flow  of  informa¬ 
tion  in  training  exercises  conducted  during  the  research  period.  This  docu¬ 
ment  presents  the  results,  conclusions,  and  implications  of  that  research. 


Olmstead,  J.  A.,  Baranick,  M.  J.,  and  Elder,  B.  L.  Research  on  training 

for  brigade  command  groups:  Factors  contributing  to  unit  combat  readi¬ 
ness.  Technical  Report  TR-78-A18,  June  1978.  (AD  A056  054) 

This  research  examined  the  relationships  between  (a)  brigade  command 
group  effectiveness  in  performance  of  Army  Training  and  Evaluation  Program 
(ARTEP)  during  Computer  Assisted  Map  Maneuver  System  (CAMMS)  exercises  and 
(b)  the  quality  of  performance  within  the  brigade  of  certain  organizational 
processes  found  in  previous  research  to  be  related  to  combat  effectiveness. 
Data  were  collected  on  11  brigades.  Seven  of  9  process  dimensions  were 
found  to  be  highly  correlated  with  Brigade  Command  Group  Effectiveness  as 
evaluated  by  exercise  controllers.  Furthermore,  large  significant  correla¬ 
tions  were  found  between  the  organizational  process  dimensions  and  a  number 
of  dimensions  subsumed  under  the  rubric  Supervision  and  Control,  which  were 
also  highly  correlated  with  Brigade  Command  Group  Effectiveness.  Implica¬ 
tions  of  the  findings  are  discussed. 


Olmstead,  J.  A.,  Baranick,  M.  J.,  and  Elder  B.  L.  A  training  feedback 
system  for  brigade  command  groups.  Technical  Report  TR-78-A19,  June 
1978.  (AD  A056  342) 

This  document  describes  a  system  for  analyzing  the  performance  of  an 
Army  brigade  command  group  during  participation  in  Computer  Assisted  Map 
Maneuver  System  (CAMMS)  exercises  and  for  providing  to  participants  system¬ 
atic  feedback  which  will  stimulate  improvement  by  the  command  group  of  its 
ability  to  meet  Army  Training  and  Evaluation  Program  (ARTEP)  standards. 


Kaplan,  I.  T.,  and  Barber,  H.  F.  Battalion  command  group  performance  in 
simulated  combat.  Technical  Paper  353,  March  1979.  (AD  A070  089) 

The  behavior  of  27  battalion  command  groups  was  studied  in  a  simulated 
combat  environment  provided  by  a  computer-driven  battle  simulator,  the  Com¬ 
bined  Arms  Tactical  Training  System  (CATTS) .  Of  the  61  subtasks  described 
in  the  Command  Group  Module  of  the  Army  Training  and  Evaluation  Program 
(ARTEP) ,  50  were  evaluated  in  the  CATTS  exercises.  Nineteen  subtasks  were 
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idantifiad  aa  relatively  weak  on  the  basis  of  their  comparatively  low  per¬ 
formance  ratings,  and  23  subtasks  were  found  to  be  highly  correlated  with 
the  overall  effectiveness  ratings.  Fourteen  subtasks  were  identified  as 
critical  because  they  were  both  low  rated  and  highly  correlated  with  over¬ 
all  effectiveness  ratings.  The  critical  subtasks  included  identifying, 
gathering,  analyzing,  and  disseminating  intelligence,  planning  fires,  de¬ 
termining  the  critical  place  and  time,  communicating  plans  and  orders,  de¬ 
feating  electromagnetic  intelligence,  reacting  to  jamming,  and  concentrat¬ 
ing  combat  power.  These  subtasks  were  related  to  more  basic  processes, 
identified  in  previous  research  as  important  determinants  of  organizational 
effectiveness,  viz.,  sensing,  decisionmaking,  communicating,  and  coping  with 
changes  in  the  environment. 

The  critical  performance  identified  in  this  report  can  be  given  par¬ 
ticular  emphasis  in  the  development  of  command  group  training  systems, 
training  programs,  and  information-processing  and  decision-aiding 
technologies. 

The  report  is  written  primarily  for  the  research  scientist  working  in 
command/control  simulation,  although  military  personnel  will  be  interested 
in  the  conclusions. 


Kaplan,  I.  T. ,  and  Barber,  H.  F.  Evaluation  of  a  computer-assisted  battle 

simulation:  CAMMS  vs  a  CPX.  Technical  Paper  355,  April  1979. 

(AD  AO 68  014) 

The  Computer-Assisted  Map  Maneuver  System  (CAMMS)  creates  a  simulated 
battle  to  train  battalion  and  brigade  command  groups  in  the  exercise  of 
command  and  control.  In  contrast  to  a  conventional  manual  command  post  ex¬ 
ercise  (CPX)  which  is  driven  by  prefabricated  messages,  OAMMS  calculates 
weapons  effects,  movement  rates,  and  logistical  support  in  real  time  to 
provide  the  command  groui  with  realistic  feedback  about  the  consequences 
of  its  actions.  The  purpose  of  this  investigation  was  to  evaluate  the  cost 
and  training  effectiveness  of  CAMMS  ir.  comparison  to  a  CPX.. 

The  players'  attitudes  toward  the  alternative  traininq  systems  were 
assessed  by  means  of  questionnaires  administered  to  50  battalion  command 
groups  end  12  brigade  command  groups  after  they  participated  in  CAMMS  ex¬ 
ercises.  Estimates  of  the  preparation  time  and  the  number  of  controllers 
required  for  each  type  of  exercise  'ere  obtained  from  14  CAMMS  exercise 
directors  and,  for  a  JPX  only,  from  5  division  and  brigade  emmanders. 

Analysis  of  the  data  showed  that  the  players  judged  CAMMS  to  be  signifi 
cantly  and  consistently  more  realistic  and  more  interesting  than  a  CPX.  The 
functional  areas  in  which  CAMMS  enjoyed  the  greatest  advantage  were  related 
to  preparing  ana  organizing  the  battlefield,  controlling  and  coordinating 
combat  operations,  and  concentrating  combat  power  as  rated  by  the  S3:  and 
in  the  exercise  of  command  and  control  rated  by  the  conuander,  especially 
exposure  to  the  capabilities  of  enemy  weapons  systems,  facing  a  thinking 
enemy,  and  making  decisions  under  real-time  constraints. 


The  principal  weaknesses  of  CAMMS  were  that  it  did  not  produce  much 
stress,  and  it  did  not  exercise  the  players  in  security  procedures,  such  as 
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electromagnetic  and  communications  security,  nor  did  it  require  them  to  re¬ 
act  to  special  situations  like  enemy  jamming,  chemical,  biological,  or 
nuclear  warfare.  The  CPX  also  received  low  ratings  in  these  areas. 

A  CAMMS  exercise  cost  25%  to  30%  less  than  a  CPX,  primarily  because 
CAMMS  required  much  less  preparation  time. 

Overall,  CAMMS  produced  a  distinctly  superior  exercise  at  a  moderate 
saving  in  cost  over  a  conventional  CPX. 

This  report  is  written  for  the  researcher  in  command/control  investi¬ 
gations,  although  military  personnel  will  be  interested  in  implications  of 
the  results. 


Barber,  H.  F.,  McGrew,  J.  F.,  Stewart,  S.  R. ,  and  Andrews,  R.  S.  The 

computer-assisted  map  maneuver  system:  A  preliminary  examination  of 
its  training  effectiveness  and  suitability  for  use  as  a  research  ve¬ 
hicle.  Research  Memorandum  79-9,  June  1979. 

The  Computer-Assisted  Map  Maneuver  System  (CAMMS)  is  a  battle  simu¬ 
lation  designed  to  train  commanders  and  staffs  of  amor,  mechanized  infantry, 
light  infantry,  and  cavalry  units  at  both  the  brigade  and  battalion  command 
levels.  The  command  groups  play  within  a  nonnuclear  environment  and  against 
a  given  enemy  force. 

This  preliminary  examination  (a)  estimated  the  training  effectiveness 
of  CAMMS,  (b)  refined  performance  measurement  procedures,  (c)  estimated  the 
feasibility  of  continuing  to  utilize  CAMMS  as  a  vehicle  for  investigating 
command  group  training.  Five  battalion  command  croups — two  mechanized  in¬ 
fantry  and  three  armor — participated.  A  pretest  and  posttest  design  was 
used. 


Based  on  the  findings,  the  researchers  concluded  that  (a)  CAMMS  showed 
evidence  of  being  an  effective  training  vehicle  for  improving  battalion  com¬ 
mand  group  proficiency,  as  subjectively  judged  in  terns  of  differential  per¬ 
formance  on  ARTEP  tasks  and  subtasks  and  an  overall  assessment  of  the  total 
command  group  and  each  major  staff  element?  (b)  development  of  a  greater 
number  of  objective  measures  of  command  group  performance  in  CAMMS  is  feasi¬ 
ble,  both  to  supplement  and  to  alternately  supplant  some  of  the  existing 
subjective  ratios;  (c)  the  relationship  of  command  group  performance  to 
battlefield  outcomes  is  complex;  (d)  performance  on  some  ARTEP  subtasks  ap¬ 
peared  to  influence  battlefield  outcomes;  (e)  organizational  process  measures 
as  used  in  this  effort  did  not  discriminate  performance  differences  among 
the  various  measures  themselves  or  change  in  performance  as  a  function  of 
the  training  exercise;  and  (f)  CAMMS  has  the  potential  for  fulfilling  the 
requirements  for  a  vehicle  training  and  training  research  for  long-term 
training  effectiveness  analysis. 

CAMMS  may  be  the  only  reasonable  vehicle  for  examining  the  integration 
of  troops  on  the  ground  with  the  play  of  a  battalion-level  command  group 
situation. 


Command  Group  Training 


Kaplan,  I.  T.,  and  Barber,  H.  F.  Training  battalion  command  groups  in  simu¬ 
lated  combat:  Identification  and  measurement  of  critical  performances. 

Technical  Paper  376,  June  1979.  (AD  A075  414) 

The  behavior  of  23  battalion  command  groups  was  investigated  in  a  simu¬ 
lated  combat  environment  provided  by  the  Combined  Arms  Tactical  Training 
Simulator  (CATTS) .  Thirteen  mechanised  groups  performed  a  covering-force 
operation  followed  by  an  attack,  and  10  nonmechanised  groups  performed  a  de¬ 
fense  and  an  attack.  Their  performance  was  rated  on  items  derived  from  the 
subtask9  of  the  battalion  command  group  ARTEP  (Army  Training  and  Evaluation 
Program).  Fifteen  subtasks  were  identified  as  critical,  because  they  or 
their  elements  were  both  low  rated  and  highly  correlated  with  ratings  of 
overall  effectiveness. 

The  four  missions  observed  in  this  investigation  were  markedly  differ¬ 
ent  with  respect  to  subtask  criticality.  All  but  1  of  the  15  critical  sub¬ 
tasks  were  identified  in  the  covering-force  mission,  5  subtasks  were  critical 
in  the  mechanised  attack,  1  in  the  defense,  and  1  in  the  nonmechanised 
attack. 

There  were  significant  differences  among  ratings  of  the  same  command 
group  by  several  observers,  each  of  whom  observed  the  exercise  from  a  dif¬ 
ferent  point  of  view.  These  results  indicate  a  need  for  further  research 
(a)  to  develop  more  objective  measures  of  performance  and  (b)  to  identify 
those  subtasks  for  which  the  different  perspectives  of  the  raters  should 
produce  valid  differences  in  performance  ratings. 

This  report  is  written  primarily  for  specialists  in  command/control 
simulation,  although  the  military  will  be  interested  in  the  results. 


Kaplan,  1.  T.  Information  flow  in  battalion  command  groups.  Technical 

Report  499,  in  press  (November  1980) . 

Information  flow  in  battalion  command  groups  was  measured  via  a  ques¬ 
tionnaire  based  on  information  presented  during  the  brigade  briefing.  Data 
from  13  groups  showed  that  a  substantial  amount  of  information  was  lost  in 
the  processes  of  communication  and  remembering.  Averaged  over  all  command 
groups,  there  were  consistent  differences  in  the  percentage  of  available 
required  information  communicated  through  particular  channels,  varying  from 
17  to  80%.  Among  battalion  commanders,  those  who  transmitted  more  informa¬ 
tion  received  less  from  their  staffs.  The  fact  that  information  loss  was 
concentrated  in  specific,  identifiable  areas,  indicates  that  efforts  to 
improve  communication  can  be  focused  where  they  are  needed. 


VI.  EXPLORATORY  RESEARCH  AND  SUPPORTING  TECHNOLOGY 
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Struh,  M.  H.  Perception  of  military  event  patterns  in  a  two-alternative 
prediction  task.  Technical  Research  Note  221,  February  1970. 

(AD  707  379) 

■As  part  of  the  continuing  effort  to  provide  research  information  which 
will  facilitate  maximum  operations  effectiveness  within  command  information 
processing  systems,  the  Tactical  Operations  System  (TOS)  Work  Unit  undertook 
a  study  of  perception  of  military  event  patterns.  The  focus  of  the  study 
was  the  recognition  of  complex  cues  to  enemy  action.  Objectives  of  the  in¬ 
vestigation,  reported  in  the  present  publication,  were  to  determine  the  role 
of  (a)  different  forms  of  second-order  patterns,  (b)  second-order  pattern 
strength,  (c)  continuity,  and  (d)  experience  as  factors  in  the  recognition 
of  second-order  patterns.  In  the  experiment,  48  enlisted  men  viewed  sequen¬ 
ces  of  military  events  representing  8  patterns  (form) ,  high  and  low  pattern 
strength,  continuous  versus  discrete  occurrence,  and  experience  (from  1  to 
10  100-trial  periods)  presented  in  systematic  design  on  CRT  screens.  The 
subjects  participated  in  two  experimental  sessions,  each  consisting  of  five 
100-trial  periods  with  10-minute  interval  breaks.  Each  man  worked  indi¬ 
vidually  and  was  self-paced.  The  men  indicated  which  of  two  enemy  activi¬ 
ties,  A(ttack)  or  R(est) ,  was  likely  to  follow  each  two  previous  events. 

They  also  responded  to  indicate  confidence  in  their  decisions.  Findings 
showed  that  given  event  patterns  occurring  with  high  frequency  (80%  of  the 
time) ,  the  decisionmaker  learned  to  predict  the  third  sequential  event  as 
often  as  it  occurred.  However,  under  conditions  of  less  frequent  pattern 
occurrence  (68%  or  less  of  the  time),  pattern  recognition  did  not  occurs 
Only  one  second-order  pattern  was  predicted  as  often  as  it  appeared.  Con¬ 
fidence  in  predicted  decisions  was  affected  by  increase  in  experience  and 
pattern  form  but  not  by  pattern  strength  nor  continuity.  It  appears  from 
the  present  findings  that  tentative  limits  are  set  on  second-order  military 
pattern  recognition.  Men  can  perceive  strong  second-order  patterns.  Com¬ 
puter  aids  would  be  quite  helpful,  however,  in  situations  where  military 
event  patterns  of  the  class  investigated  are  weak  but  substantial  enough 
to  improve  decision  accuracy. 


Johnson,  E.  M. ,  and  Halpin,  S.  M.  Preliminary  evaluation  of  a  multistage 
Bayesian  inference  system.  Proceedings  of  the  1972  International  Con- 
*nce  on  Cybernetics  and  Society.  New  York:  IEEE,  1972,  431-435. 

The  inferential  leap  made  by  decisionmakers  in  going  from  data  set  to 
hypothesis  set  is  a  major  problem  in  the  development  of  computer-aided  in¬ 
ference  systems.  A  potential  solution  to  this  problem  is  the  use  of  multi¬ 
stage/hierarchical  inference  techniques.  A  multistage  Bayesian  inference 
system  has  been  developed  and  an  on-line  test-bed  being  used  to  develop 
and  evaluate  related  concepts  is  presented.  Tentative  findings  from  an  on¬ 
going  exploratory  study  are  discussed. 
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Halpin,  S.  M. ,  Johnson,  E.  M. ,  and  Thornberry,  J.  A.  Cognitive  reliability 
in  manned  systems.  IEEE  Transactions  on  Reliability ,  1973,  R-22, 
165-170. 

Human  components  in  manned  systems  often  compensate  for  hardware  error 
by  utilizing  other  information  and  past  experience  in  addition  to  the  normal 
hardware  output.  However,  errors  in  human  information  processing  and  utili¬ 
zation,  cognitive  reliability,  often  lowers  the  overall  system  reliability. 
Cognitive  reliability  in  manned  systems  is  discussed  in  terms  of  tl  .j  types 
of  human  errors  which  may  occur  and  in  terms  of  factors  which  affect  the 
occurrence  of  these  errors.  It  is  a  complex  function  of  attitudir.al  and 
structural  factors  and  their  interaction.  Cognitive  reliability  is  useful 
for  examining  man's  role  in  complex  systems  where  cognitive  as  well  as 
perceptual-motor  skills  are  required. 


Johnson,  E.  M. ,  Cavanagh,  R.  C.,  Spooner,  R.  L. ,  and  Samet,  M.  G.  Utiliza¬ 
tion  of  reliability  measurements  in  Bayesian  inference:  Models  and 
human  performance.  IEEE  Transactions  on  Reliability,  1973,  R-22, 
176-183.  "  "  '  ‘ 

A  normative  two-stage  model  for  incorporating  reliability  measurements 
of  data-reporting  sources  in  a  Bayesian  inference  system  is  presented.  An 
experiment  required  human  subjects  to  make  intuitive  inferences  about  two 
hypotheses  on  the  ba.;is  of  sample  data  which  were  reported  with  a  given  re¬ 
liability.  When  compared  with  the  optimal  model,  subjects  exhibited  sys¬ 
tematic  errors  in  estimating  the  diagnostic  impact  of  less  than  perfectly 
reliable  data.  Their  responses  reflected  the  use  of  specific  nonoptimal 
heuristic  strategies  to  process  the  information.  A  utility  function  was 
added  to  the  normative  model  to  illustrate  how  a  best  choice  might  be  made 
from  among  potential  data-gathering  experiments  whose  costs  increase  with 
their  reliabilities.  Recommendations  for  using  computer  aids  to  enhance 
efficiency  in  inference  systems  are  made. 


Levine,  J.  M. ,  and  Samet,  M.  G.  Information  seeking  with  multiple  sources 

of  conflicting  and  unreliable  information.  Human  Factors,  1973,  15 , 

407-419. 

The  effects  of  information  conflict,  the  frequency  with  which  the  infor¬ 
mation  is  updated,  and  the  reliability  of  the  information  sources  upon  infor¬ 
mation  seeking  and  decision  behavior  were  studied.  Sixteen  U.S.  Army  en¬ 
listed  men  performed  a  computer-controlled  task  in  which  they  had  to  request 
updated  enemy-position  reports  from  three  information  sources  to  determine 
which  of  eight  locations  was  the  target  of  a  gradual  enemy  advance.  Forty 
problems,  presented  twice  each,  were  arranged  in  two  within-subject  factorial 
designs — with  source  reliability  varied  between  and  within  problems,  re¬ 
spectively.  Subjects  were  permitted  to  make  up  to  three  decisions  on  each 
problem,  with  correct  decisions  rewarded  and  incorrect  decisions  penalized 
according  to  a  nonlinear  payoff  function.  Results  from  analyses  of  variance 
indicated  that  less  information  was  sought  prior  to  an  initial  decision  ta)  for 
higher  than  for  lower  reliability  conditions,  (b)  as  update  frequency  de¬ 
creased,  and  (o)  as  degree  of  conflict  increased.  When  all  sources  were  of 
higher  reliability,  accuracy  of  decisions  was  higher;  but,  in  general,  accu¬ 
racy  was  complexly  affected  by  conflict  and  update  frequency.  These  results 
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were  discussed  in  terms  of  the  cost  of  requesting  information  and  the  proba¬ 
bility  that  an  update  would  contain  accurate  information.  Information  sampling 
and  decision  latencies  as  well  as  other  indices  of  performance  were  also  evalu¬ 
ated  and  discussed. 


Baker,  J.  D.,  and  Johnson,  E.  M.  (Eds.)  Field  testings  The  delicate  com¬ 
promise.  Special  Issue  of  Human  Factors,  June  1974,  16,  203-214. 

inis  paper  >  electively  compares  human  factors  field  testing  with  labora- 
uo'.  orientation .  Although  each  is  derived  from  the  scientific  method, 

there  are  critical  differences  between  the  two  research  settings.  Field 
ter ting  is  not  a  simple  extension  of  the  laboratory  into  an  operational 
setting.  Techniques,  procedures,  and  research  strategies  differ;  a  tech¬ 
nology  is  evolving  to  bridge  the  gap  between  the  two  research  domains. 


Levine,  J.  M.,  and  Samet,  M.  G.  Information  seeking  with  input  pacing  and 

multiple  decision  opportunities.  Human  Factors,  1974,  16,  384-394. 

Forty-eight  college  students  performed  a  task  in  which  they  had  to  re¬ 
quest  updated  enemy  position  reports  from  three  information  sources  to  de¬ 
termine  which  of  eight  locations  was  the  target  of  a  gradual  enemy  advance. 
Eight  different  problems,  presented  twice  each,  were  arranged  into  two  fac¬ 
torial  designs.  In  Design  FP,  the  effects  of  pacing  rate,  pacing  variability, 
and  the  number  of  decision  modification  opportunities  were  evaluated.  In 
Design  SP,  information  requests  were  self-paced  rather  than  f orced-paced ; 
the  effects  of  the  number  of  decision  modification  opportunities  and  basis 
of  payoff  were  studied.  Results  indicated  that  (a)  self  pacing  and  fast 
forced  pacing  resulted  in  more  information  seeking  and  greater  accuracy  than 
did  slow  forced  pacing,  and  (b)  increased  opportunities  for  decision  modifi¬ 
cation  generally  decreased  the  accuracy  of,  and  confidence  in,  first  decisions. 
The  rate  at  which  information  was  presented  was  more  important  than  whether  it 
was  provided  automatically  or  upon  request. 


Johnson,  E.  M. ,  and  Halpin,  S.  M.  Multistage  inference  models  for  intelli¬ 
gence  analysis.  Proceeding  of  the  1974  Army  Science  Conference.  U.S. 
Military  Academy,  West  Point,  N.Y. ,  18-21  June  1974. 

The  objective  of  intelligence  analysis  is  to  impose  a  consistent  and 
coherent  interpretation  upon  seemingly  unrelated  or  even  conflicting  bits 
and  pieces  of  information,  i.e.,  to  transform  information  into  intelligence. 
A  lack  of  structure  and  the  frequent  absence  of  a  logical  relation  between 
the  available  information  and  potential  enemy  capabilities  make  that  task 
intellectually  complex  and  demanding.  Multistage  inference  models  provide 
a  potentially  meaningful  and  useful  aid  for  intelligence  analysis.  Multi¬ 
stage  Bayesian  inference  models  are  described  and  their  derivation  outlined. 
These  models  are  based  on  a  decomposition  of  the  conditional  relationship 
between  a  data  set  and  an  hypothesis  set  into  a  logical  sequence  of  inter¬ 
mediate  sets  linking  the  data  set  to  the  terminal  hypothesis  set.  These 
models  describe  optimal,  normative  inference  strategies  which  are  analogous 
to  the  intuitive  procedures  currently  used  by  intelligence  analysts,  but 
which  facilitate  the  integration  of  historical  data  and  expert  opinion  into 
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a  formal  inference  model  wnile  reducing  the  cognitive  demands  on  the  analyst. 
Several  experiments  on  human  performance  in  multistage  inference  tasks  are 
presented.  These  experiments  compared  intuitive  (unaided)  inferences  and 
inference  strategies  with  the  normative  multistage  model,  and  examined  in¬ 
telligence  officer's  performance  with  and  reaction  to  an  on-line  aid  based 
on  a  multistage  model. 


Johnson,  E.  M.  The  effect  of  data  source  reliability  on  intuitive  infer¬ 
ence.  Technical  Paper  251,  July  1974.  (AD  784  097) 

As  part  of  a  continuing  effort  to  provide  research  information  which 
will  facilitate  improved  intelligence  information  processing,  the  Intelli¬ 
gence  Information  Processing  Program  undertook  a  study  of  mam's  processing 
and  utilization  of  unreliable  data.  Reports  from  data  sources  of  given  re¬ 
liability  and  diagnosticity  were  presented  to  22  subjects  in  a  series  of  two 
hypothesis  decision  problems.  On  each  problem,  each  subject  indicated  the 
most  likely  of  the  two  hypotheses  amd  the  subjective  odds  favoring  that  hy¬ 
pothesis.  Subjective  odds  varied  as  a  function  of  the  data  diagnosticity 
and  source  reliability.  However,  the  subjects  generally  failed  to  extract 
as  much  certainty  as  possible  from  the  data — subjective  odds  were  generally 
conservative  with  respect  to  odds  computed  by  a  normative  Bayesian  model. 
However,  in  most  cases,  as  reliability  decreased,  subjective  odds  increased 
relative  to  Bayesian  odds  until  they  were  generally  greater  than  Bayesian 
odds  at  the  lowest  level  of  reliability .  Subjects'  protocols  and  data 
analyses  indicated  that  subjects  were  using  nonoptimal  inference  strategies 
in  which  reliability  was  incorporated  as  a  multiplicative  weighting  factor. 
If  the  diagnostic  impact  of  the  data  "if  it  were  true"  is  correctly  evalu¬ 
ated,  this  strategy  leads  to  increasingly  inaccurate  subjective  odds  as  re¬ 
liability  decreases  and  the  data  diagnosticity  increases. 


Kneppreth,  N.  P.,  Gustafson,  D.  H.,  Leifer,  R.  P. ,  and  Johnson,  S .  M.  Tech¬ 
niques  for  the  assessment  of  worth.  Technical  Paper  254,  August  1974. 

(AD  784  629) 

Subjective  worth  forms  the  basis  for  the  selection  of  future  alternatives 
and  the  evaluation  of  past  actions.  During  the  past  few  years  several  formal 
methods  of  worth  assessment  have  been  developed.  These  methods  provide  an 
explicit  basis  for  incorporating  military  judgment  into  the  assessment  pro¬ 
cess  in  a  manner  designed  to  produce  consistent  and  valid  assessment.  Seven 
general  methods  for  worth  assessment  are  reviewed.  Step-by-step  procedures, 
a  specific  example,  and  variations  of  each  method  are  provided.  The  descrip¬ 
tion  of  each  method  includes  training  and  equipment  requirements,  poss  ble 
problems,  type  of  results,  the  advantages  and  disadvantages,  methodr  .or 
checking  accuracy,  and  sources  of  information.  The  methods  cover  four  areas 
of  assessments  those  which  yield  worth  on  an  ordinal  scale,  those  which 
yield  numerical  values  directly,  the  gamble  methods  involving  probabilities, 
and  the  multivariate  methods  for  dealing  with  multiple  or  dependent  factors. 
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Rapoport,  A. ,  Burkheimer,  G.  J.,  and  Stein,  W.  E.  Studies  of  detection  of 

change  processes.  Research  Memorandum,  Draft,  November  1974. 

This  report  considers  a  two-state  discrete-time  Markov  decision  process 
with  partially  observable  states.  The  process  occurs  in  equipment  checking, 
intelligence  gathering,  target  search,  signal  detection,  medical  diagnosis, 
and  reaction  time  studies.  Informally,  it  is  characterized  by:  (a)  a  par¬ 
tially  observable  state  Sq  that  at  some  stage,  which  is  determined  proba¬ 
bilistically  and  independently  of  the  decisionmaker's  (DM)  actions,  turns 
into  another  partially  observable  state  Si  and  stays  there  permanently; 

(b)  observations  that  are  taken  sequentially,  one  on  each  stage,  and  that 
provide  DM  imperfect  information  about  the  true  state;  (c)  a  binary  decision 
on  each  stage:  C — the  change  has  occurred  (i.e.,  Si  is  the  true  state), 
or  W— continue  (i.e.,  take  one  more  observation);  (d)  an  appropriate  cost 
structure,  which  reflects  both  the  decision  made  and  whether  or  not  it  is 
correct;  and  (e)  a  well-informed  DM  who  attempts  to  minimize  the  expected 
loss  of  the  process. 

Two  variants  of  the  process  are  presented  and  discussed  in  chapter  1. 

The  optimal  decision  policy,  which  minimizes  expected  loss,  is  presented  and 
examined  for  both  variants  in  chapter  2.  Three  deterministic  response  models 
are  formulated  and  investigated  in  chapters  3,  4,  and  5.  The  first,  model  I, 
prescribes  a  C  decision  as  soon  as  DM's  probability  that  the  change  has  oc¬ 
curred  exceeds  a  predetermined  probability  boundary.  According  to  model  II, 
a  C  decision  is  made  as  soon  as  a  predetermined  number  of  observations,  k, 
greater  than  or  equal  to  a  fixed  critical  observation  value  have  been  taken. 
Model  III  is  like  II  with  the  only  exception  that  the  k  observations  must 
be  successive.  Testable  predictions  are  derived  for  all  three  models,  in¬ 
cluding  the  probability  of  an  incorrect  C  decision  (false  alarm) ,  the  proba¬ 
bility  distributions  of  number  of  observations  taken  conditional  on  the 
starting  or  terminal  state,  and  the  probability  distribution  of  number  of 
observations  after  the  change. 

The  sensitivity  of  the  optimal  model  to  perturbations  ir.  the  parameter 
values  is  examined  in  chapter  6.  A  systems  operating  characteristic  (SOC) 
curve  is  proposed  to  assess  the  sensitivity  of  the  optimal  policy  and  to 
compare  the  throe  models  to  one  another.  The  optimal  model  as  well  as  the 
three  models  are  tested  in  a  computer-controlled  detection  of  change  experi¬ 
ment  in  which  seven  paid  subjects  participated  individually  in  sixteen 
3-hour  sessions.  A  modification  of  model  I,  which  allows  for  problem-to- 
problem  learning,  is  proposed  to  best  account  for  the  data. 

The  two-state  process  is  extended  to  the  multistate  case  in  chapter  8. 
The  optimal  policy  for  the  general  case  is  presented,  several  cases  with 
three  states  are  examined  in  detail ,  and  several  numerical  examples  are 
presented  and  discussed.  Concluding  remarks  are  presented  in  the  final 
chapter . 


Pask,  G.  Current  approaches  to  decision  making  in  complex  systems.  Tech¬ 
nical  Report  TR-76-B1,  April  1976. 

In  July  1975,  a  conference  was  held  at  Richmond,  Surrey,  England  to 
elicit  a  fair  picture  of  the  state-of-the-art  in  decisionmaking  in  Europe; 
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the  status  of  decision  oriented  disciplines  and  ongoing  or  contemplated 
lines  of  research.  Twelve  papers  on  the  subject  of  human  decisionmaking 
were  presented  and  discussed. 

1.  The  otherwise  confusing  picture  of  human  decisionmaking  is  some¬ 
what  simplified  by  the  existence  of  individually  and  culturally 
consistent  cognitive  styles  and  methods  of  recall. 

2.  It  is  possible  that  people  may  be  trained  to  respond  in  a  particu¬ 
lar  way  that  yields  performance  data:  for  example,  to  use  confi¬ 
dence  estimation  and  rating  facilities.  However,  this  does  not 
mean  that  they  can  be  trained  to  decide. 

3.  Attempts  to  fit  simple  models  to  human  performance  are  generally 
unsuccessful,  especially  for  sequential  and  temporally  dependent 
choice.  More  predictive  models  are  available  at  the  price  of  in¬ 
creased  sophistication. 

4.  Decision  analysis  is  not  a  prescriptive  exercise,  introducing  the 
decisionmaker  to  what  he  should  do.  Nor,  e. s  sometimes  supposed, 
is  it  descriptive.  The  chief  merit  of  the  method  is  to  establish 
a  compromise  between  the  decisionmaker  and  the  expert.  It  was 
suggested  that  man/machine  systems  could  be  analyzed  and  designed 
from  a  similar  point  of  view. 


Johnson,  E.  M. ,  and  Huber,  G.  P.  The  technology  of  utility  assessment. 

IEEE  Transactions  on  systems,  man  and  cybernetics,  1977,  SMC- 7 ,  311-325. 
(Also  in  Sage,  A.  P.  (Ed.)  Readings  in  Systems  Engineering  Methodology 
and  Applications.  New  York:  IEEE  Press,  1977.) 


Utility  assessment  includes  the  entire  process  of  identifying,  measur¬ 
ing,  and  combining  attributes  to  create  an  explicit  value  structure  that 
can  form  a  basis  for  evaluations  and  decisions.  The  utility  assessment 
process  is  reviewed,  and  selected  issues  and  problems  are  discussed.  The 
focus  is  on  applications  in  field  and  field-like  settings  where  it  is  neces¬ 
sary,  in  order  to  solve  a  problem,  to  assess  the  utility  of  items  or  alter¬ 
natives  having  more  than  one  valued  property.  Although  some  aspects  of  the 
paper  are  new,  much  of  the  paper  is  tutorial  in  nature.  Assessment  tech¬ 
niques  are  discussed  with  particular  emphasis  on  the  factors  which  determine 
the  relative  effectiveness  of  the  various  techniques  in  specific  situations. 
Assessment  models  are  discussed,  along  with  a  description  of  the  areas  where 
they  have  been  applied.  Guidelines  are  developed  for  improving  the  proba¬ 
bility  of  success  in  applying  utility  assessment,  and  suggestions  made  for 
the  research  necessary  to  further  improve  the  technology  of  utility  assessed. 


Kneppreth,  N.  P.,  Hoessel,  W.  H. ,  Gustafson,  D.  H.,  and  Johnson,  E.  M.  A 
strategy  for  selecting  a  worth  assessment  technique.  Technical  Paper 
280,  February  1978.  (AD  A055  345) 

A  variety  of  techniques  exist  for  evaluating  and  assessing  subjective 
worth.  This  paper  develops  and  presents  a  strategy  for  selecting  the  most 
appropriate  worth  assessment  technique  for  a  given  situation. 
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A  three-round  Delphi  procedure  was  used  to  systematically  solicit  and 
collate  informed  judgments  from  practitioners  and  expertB  in  the  field  of 
worth  assessment.  The  first  Delphi  questionnaire  identified  41  factors  in¬ 
fluencing  the  selection  and  successful  use  of  worth  assessment  techniques. 

The  second  Delphi  questionnaire  was  used  to  determine  the  relative  impor¬ 
tance  of  the  41  factors  identified  in  the  first  round  of  the  Delphi.  Ten 
composite  factors,  four  dichotomous  and  six  continuous,  were  developed  which 
appeared  to  account  for  the  41  factors  and  their  relative  weights.  The 
third  Delphi  questionnaire  was  used  to  develop  a  profile  for  nine  different 
worth  assessment  techniques  on  the  six  continuous  factors.  A  three-step 
strategy,  based  on  the  10  composite  factors,  was  developed  for  selecting  a 
specific  worth  assessment  technique  for  a  given  situation.  The  strategy 
developed  for  selecting  a  worth  assessment  technique  should  serve  to  increase 
the  successful  application  of  worth  assessment  techniques  to  specific  situa¬ 
tions,  such  as  occur  in  the  appraisal  of  intelligence  information. 


Goodman,  B.,  Fischhoff,  B.,  Lichtenstein,  S.,  and  Slovic,  P.  The  training 
of  decision  makers.  Report  of  a  Conference  held  at  the  Oregon  Research 
Institute.  Technical  Report  TR-78-B3 ,  March  1978. 

The  past  15  years  have  seen  important  advances  in  the  understanding  of 
behavioral  decision  processes.  Despite  these  advances  there  has  been  sur¬ 
prisingly  little  attention  paid  to  the  training  of  decisionmakers.  This 
report  provides  a  summary  of  a  conference  held  in  July  1976  in  Eugene, 

Oregon,  to  evaluate  the  state  of  knowledge  regarding  the  training  of  de¬ 
cisionmakers,  identify  important  research  questions  and  recommend  directions 
for  future  research.  The  report  provides  both  a  brief  review  of  current 
training  programs,  including  a  discussion  of  key  attitudes  and  skills  re¬ 
quired  for  decisionmaking ,  and  a  list  of  the  most  important  research  questions 
that  emerged  during  the  conference;  three  aspects  of  the  training  problem 
recommended  for  immediate  attention  were  (1)  training  specific  skills, 

(2)  evaluating  the  quality  of  decisions,  and  (c)  implementing  the  knowledge 
obtained  through  decision  research. 


Pask,  G.  (Ed.)  Current  approaches  to  decision  making  in  complex  systems 

II.  Technical  Report  TR-78-B4,  March  1978. 

The  proceedings  of  the  Second  (1976)  Richmond  Conference  upon  "Current 
approaches  to  Decision  Making  in  Complex  Systems"  held  at  Richmond  are  re¬ 
ported.  The  report  follows  an  earlier  report  "Current  Scientific  Approaches 
to  Decision  Making  in  Complex  Systems",  the  proceedings  of  the  First  (1975) 
Conference  on  the  theme,  issued  as  ARI  Technical  Report  TR-76-B1. 

The  following  main  conclusions  we»e  reached:  (a)  Some  individuals  at 
least  can  be  treated  as  decisionmakers.  (b)  There  is  no  uniquely  "best" 
decision  training  method  but  quite  a  lot  can  be  said  about  techniques  that 
do  not  work  on  their  own  (though  they  may  contribute  to  a  training  process) . 
(c)  Decisionmaking  is  probably  a  collection  of  skills  integrated  in  the 
framework  of  tasks  between  which  there  is  adequate  generalization,  (d)  In 
specific  situations  decision  skill  can  be  improved,  (e)  Decisionmakers  must 
learn ,  in  concepts,  facts,  principles,  attitudes,  procedures  and  heuristics, 
(f)  "Learning  to  Learn"  applies  to  a  decision  situation  (or  class  of 
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situations) .  (g)  The  "optimum  balance"  of  aiding  against  training  appears 

to  be  task  dependent  and  dependent  upon  idiosyncracies  of  decision  style. 

(h)  Several  coherent  structuring  schemes  were  discussed. 

The  following  issues  were  regarded  by  most  participants  as  inadequately 
explored  by  current  empirical  studies  and  theoretical  developments:  (i)  Pre¬ 
scriptions  for  training  that  involve  problem  formulation  and  recognition  in 
large  but  specific  task  settings.  (j)  Logics  of  action.  (k)  structuring, 
both  internal  and  external. 


Smith,  D.  E.,  and  Gardner,  R.  L.  An  approach  for  providing  more  accurate  V 

probability  assessments.  Technical  Report  TR-78-B7,  May  1978. 


This  report  describes  a  possible  approach  for  providing  more  accurate 
probability  assessments  in  those  real-world  decision-making  situations  for 
which  relative  frequency  information  generally  does  not  exist.  In  this  ap¬ 
proach  an  estimated  functional  relationship  is  obtained  from  a  comparison, 
in  relative  frequency  base  (RFB)  problems,  of  a  person's  posterior  proba¬ 
bility  assessments  with  the  true  posterior  probabilities.  Then,  the  func¬ 
tional  relationship  is  used  as  a  model  to  provide  probability  predictions 
for  new  problems  by  adjusting  the  person's  corresponding  probability  assess¬ 
ments,  Experimental  evidence  indicated  that  the  resulting  models  can  pro¬ 
vide  satisfactory  probability  predictions  for  problems  having  identical 
structure  to  those  used  in  model  construction.  At  this  time,  some  obstacles 
prevent  immediate  use  of  these  models  for  probability  prediction  in  non- 
relative  frequency  base  (NRFB)  problems.  However,  it  would  be  unwise  to 
discount  the  feasibility  of  developing  a  practical  RPB-based  model  for  ad¬ 
justing  real-world  probability  estimates  until  the  approach  were  tested  with 
participants  and  problems  in  a  field  of  common  expertise  (such  as  intelli¬ 
gence  analysis  or  weather  forecasting,  for  example)  where  more  uniformity 
of  background  and  experience  exists . 


Negroponte,  N.,  Herot,  C.,  and  Weinzapfel,  G.  One-point  touch  input  of 

vector  information  for  computer  displays.  Technical  Report  TR-78-TH3 , 
October  1978.  (AD  A064  278) 

The  finger  as  a  graphical  stylus  enjoys  a  coefficient  of  friction  with 
glass  sufficient  to  provide  input  of  direction  and  torque  as  well  as  position 
from  a  single  point.  This  report  describes  a  pressure  sensitive  digitizer 
(PSD)  capable  of  accepting  these  force  inputs,  and  discusses  a  set  of  five 
simple  input  applications  used  to  assess  the  capabilities  of  this  device. 
These  applications  include  techniques  for  specifying  vectors,  and  pushing, 
pulling,  dispersing,  and  reorienting  objects  with  a  single  touch.  Experience 
gained  from  these  applications  demonstrates  that  touch  and  pressure  sensing 
open  a  rich  channel  for  immediate  and  multidimensional  interaction. 
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The  project  is  an  ex  uuple  of  basic  research  with  potential  for  future 
application.  The  report  will  be  of  interest  primarily  to  research  scien¬ 
tists  in  the  field  of  graphic  CRT  displays. 
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Lichtenstein,  S.,  and  Pischhoff,  B.  Training  for  calibration.  Technical 

Report  TR-A78-A32,  November  1978.  (AD  A069  703) 

Two  experiments  attempted  to  improve  the  quality  of  people's  probability 
assessments  through  intensive  training.  The  first  involved  11  sessions  of 
200  assessments  each  followed  by  comprehensive  feedback.  It  showed  consid¬ 
erable  learning,  almost  all  of  which  was  accomplished  after  receipt  cf  the 
first  feedback.  There  was  modest  generalization  to  several  related  proba¬ 
bility  assessment  tasks,  no  generalization  at  all  to  two  others.  The  second 
experiment  reduced  the  training  to  three  sessions.  It  revealed  the  same 
pattern  of  learning  and  limited  generalization.  The  implications  of  these 
results  are  discussed. 


Bloomfield,  J.  R. ,  Beckwith,  W.  E.,  Emerick,  J.,  Marmurek,  H.  H.,  Ebo  Tei,  B. 

and  Traub,  B.  H.  Visual  search  with  embedded  targets.  Technical  Re¬ 
port  TR-78-TH8,  December  1978.  (AD  A069  666) 

Visual  search  may  be  necessary  where  nontarget  objects  in  the  search 
area  compete  for  attention  with  the  target  object  (competition  situation) 
or  where  a  target  blends  with  its  background  (embedded  situation) .  This 
research  is  the  first  direct  investigation  of  parameters  of  embedded  target 
search  situations.  The  research  sought  (a)  to  determine  the  relationships 
between  measures  of  visual  search  performance,  peripheral  visual  acuity, 
and  ratings  of  discriminability  obtained  with  embedded  targets,  and  (b)  to 
compare  competition  and  embedded  search  tasks. 

Competition  tasks  used  constructed  color  displays;  embedded  tasks  used 
vs  different  color  and  four  different  textural  black-and-white  displays.. 

1  >edded  target  discriminability  was  rated  by  28  observers  for  all  5  colors 
<  ’  4  textures,  in  3  different  target  locations.  Several  sets  of  visual 
s  rch  experiments  were  carried  out.  In  2  experiments  using  embedded  target 
displays,  search  times  were  recorded  for  the  5  color  targets  and  the  4  black- 
and-white  textural  targets.  In  4  experiments,  with  the  elements  of  the 
colo  display  separated  to  varying  degrees  to  create  competition  search 
task-,  search  times  were  recorded,  for  a  target  that  differed  in  size  (1 
exrr>riment)  and  for  2  of  the  color  targets  (3  experiments) .  Peripheral 
vis.~.i  acuity  was  measured  for  the  color  embedded  display,  the  color  sepa¬ 
rated  displays,  and  the  black-and-white  embedded  display. 

Search  time  was  inversely  proportional  to  peripheral  acuity  and  to  dis¬ 
criminability.  Embedded  search  was  easier  than  competition  search.  In  com¬ 
paring  competition  and  embedded  target  search  tasks,  two  opposing  effects 
were  noted.  As  the  separation  of  the  background  elements  increased,  the  tar¬ 
get  became  harder  to  discriminate,  tending  to  increase  search  time,  but  the 
number  of  background  elements  present  within  the  fixed  display  area  decreased 
tending  to  decrease  search  time.  The  order  of  difficulty  of  the  two  color 
targets  used  changed  in  going  from  the  embedded  to  the  competition  task. 

This  report  shows  that,  using  complex  texture  backgrounds,  simple  rela¬ 
tionships  exist  between  visual  search  time,  peripheral  visual  acuity,  and 
rated  discriminability.  The  technological  base  research,  of  interest  pri¬ 
marily  to  research  specialists,  suggests  that  further  study  may  lead  to  the 
development  of  predictive  procedures  for  search  performance  in  complex  real 
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world  tasks  (such  as  air-to-ground  target  acquisition) ,  and  could  give  us 
the  ability  to  select  and  train  effective  observers  for  these  search  tasks. 


Bonnet,  D.  G. ,  and  Synder,  H.  L.  Prediction  of  the  recognition  of  real  ob¬ 
jects  as  a  function  of  photometric  and  geometric  characteristics. 

•  Technical  Report  TR-78-TH7,  December  1978.  (AD  A071  118) 

The  purpose  of  this  research  was  to  predict  target-by-target  acquisition 
performance  of  air-to-ground  imagery  from  microdensitometrically  determined 
photometric  and  geometric  characteristics  of  the  scene.  Results  showed  that 
it  is  feasible  to  predict  the  ground  range  at  which  a  given  target  will  be 
detected  by  ait  airborne  observer.  This  prediction  could  be  made  totally 
automatically,  given  reconnaissance  imagery,  a  microdensitometer,  and  a 
small  computer. 

Seventeen  characteristics  of  targets,  backgrounds,  and  target/background 
relationships  which  reliably  correlate  to  target  acquisition  performance  were 
identified.  Sixty  regression  equations  combining  these  variables  into  linear 
predictive  models  of  target  acquisition  performance  were  developed  from  one 
set  of  targets  and  mission  conditions  and  cross-validated  against  targets 
contained  in  three  different  reconnaissance  missions. 

The  best  prediction  is  obtained  when  at  least  two  orthogonal  scans  are 
passed  through  the  target  of  at  least  two  frames  of  the  reconnaissance  film. 
With  three  properly  weighted  predictor  variables  derived  from  these  scams, 
up  to  92%  of  the  variance  in  target  detectability  is  predicted.  The  predic¬ 
tion  equation  contains  one  measure  of  target  size,  one  of  background  hetero¬ 
geneity,  and  one  of  target/background  contrast.  Performance  is  most  predict¬ 
able  when  the  minimum  available  range  is  taken  as  the  criterion  in  the  event 
of  am  incorrect  response. 

The  report  is  of  primary  interest  to  research  personnel. 


Shaket,  E.,  Ben-Bassat,  M.,  Madni,  A.,  and  Leal,  A.  Applications  of  adap¬ 
tive  programming  technology  to  commamd  group  training  and  performance 
improvement.  Technical  Report  TR-79-A22.  May  1979.  (AD  A076  234) 

This  report  provides  a  feasibility  analysis  and  evaluation  of  the  appli¬ 
cation  of  Adaptive  Programming  Technology  (APT)  to  the  improvement  of  command 
group  performance.  The  final  high  payoff  application  area  recommended  is  a 
situation  assessment  aid  for  a  division-level  G2.  APT  technology  is  expected 
to  provide  a  concept-oriented,  interactive  aid  to  integrate  the  large  informa¬ 
tion  volume  available  into  a  coherent  global  situation  assessment.  A  5-year 
development  program  is  provided. 


Badre,  A.  N,  Selecting  and  representing  information  structures  for  battle¬ 
field  information  systems.  Technical  Report  TR-79-A20,  June  1979. 

(AD  A071  117) 

The  objective  of  this  research  was  to  identify  and  apply  experimental 
techniques  for  locating  and  evaluating  data  patterns  and  informational  chunks 


Vl-io 


Technology  Base 


which  are  meaningful  to  the  battlefield  commander  in  his  decision-making 
tasks.  The  identification  of  meaningful  chunks  of  information  is  useful  in 
specifying  criteria  for  the  development  of  decision-aiding  algorithms  that 
search  for,  classify,  and  display  information.  Two  main  tasks  were  used  for 
collecting  data  about  informational  chunks)  reconstruction  and  copying.  In 
the  reconstruction  task  an  experimental  run  consisted  of  a  successive  num¬ 
ber  of  trials.  In  each  trial  the  participant  first  viewed  a  battlefield  map 
scenario  for  either  10  seconds  or  1  minute  after  which  it  was  removed  and  he 
was  asked  to  reconstruct  it.  In  the  copying  task,  the  participant  was  asked 
to  copy  the  scenario  as  rapidly  and  as  accurately  as  he  could.  As  the  par¬ 
ticipant  reconstructed  or  copied  the  unit  symbols  of  each  scenario,  the 
experimenter  recorded  both  the  order  and  ti's  of  symbol  placements.  Two 
entirely  different  criteria  were  used  to  partition  the  reconstructed  sce¬ 
narios  into  chunks  of  symbols.  These  were  the  interplacement  times  criterion 
(IPT) ,  which  is  based  on  computing  the  average  placement  times  for  the  copy¬ 
ing  task;  and  the  sequence  of  tactically  related  symbols  (sequential)  derived 
from  a  predetermined  set  of  tactically  meaningful  relations.  Both  criteria 
proved  useful  in  evaluating  participant  use  of  information  chunks. 


Lippman,  A.,  and  Negroponte,  N.  Graphical  input  techniques.  Technical  Re¬ 
port  409,  September  1979.  (AD  A075  427) 

A  graphic  work  station  has  been  created  where  three  aspects  of  mem- 
machine  interaction  may  be  tested:  (a)  interaction  with  a  flat  display; 

(b)  coplanar  input  and  output;  and  (c)  the  ability  to  "see  through  your 
hand"  to  a  display  below.  All  three  are  created  by  a  suitable  arrangement 
of  mirrors  to  direct  a  projected  television  image  to  a  back-projection  screen 
which  is  in  turn  optically  superimposed  onto  a  data  tablet  surface  via  a 
semitransparent  mirror.  Uses  and  benefits  of  the  device  are  demonstrated  by 
means  of  a  set  of  games  which  simulate  a  situation  where  a  user  must  inter¬ 
act  with  dynamically  changing  data  and  imagery,  and  annotation  programs 
where  rapid  data  entry  from  hard  copy  is  required. 


Halpin,  S.  M.  Utilization  of  partially  accurate  output  from  a  computer  aid. 

Working  Paper  HF-80-5,  November  1979. 

This  paper  provides  a  compilation  of  research  conducted  at  ARI  during 
1972-74  on  the  issue  of  the  credibility  of  computer  output  as  a  decision  aid. 
This  research  addresses  itself  to  a  concern  among  those  engaged  in  the  de¬ 
velopment  of  computer  support  of  decisionmaking,  this  being  the  extent  to 
which  the  intended  user  will  actually  benefit  from  the  aid.  Results  from 
research  done  in  the  area  have  shown  that  even  those  who  have  more  or  less 
grown  up  with  computers  fail  to  take  full  advantage  of  the  tools  provided 
by  system  developers. 

The  first  part  of  this  paper  (chapter  1)  presents  an  overview  of  re¬ 
search  presented  to  Division  21  at  the  American  Psychological  Association 
meeting,  1974.  The  second  part  of  the  paper  (chapters  2  and  3)  p^usents 
detailed  descriptions  of  the  individual  experiments. 
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Pask,  G. ,  and  Robinson,  M.  Current  approaches  to  scientific  decision  making 
in  complex  systems  Ill.  Vol.  1:  Final  technical  report.  Research  Note 
80-10,  January  1980. 

The  main  theme  of  the  Third  Richmond  Conference  was  the  area  of  founda¬ 
tional  problems  in  decisionmaking.  The  conference  is  reported  in  two  volumes. 
Volume  II  contains  the  major  papers  presented  at  the  conferences  Observable 
Components  of  the  Decision  Process  and  a  Revised  Theoretical  Position,  Gordon 
Pask;  Decisionmaking  rs  an  Event-search:  Traffic  on  a  Multidimensional  Struc¬ 
ture,  R.  H.  Atkin;  Decision:  Foundation  and  Practice,  Brian  R.  Gaines;  Com¬ 
peting  Modes  of  Cognition  and  Communication  in  Simulated  and  Self-Reflective 
Systems,  Stein  Broten;  On  the  Spontaneous  Emergence  of  Decision  Making  Con¬ 
straints  in  Communicating  Hierarchical  Systems,  John  S.  Nicolis.  Volume  1 
contains  an  overview  of  each  paper,  the  author's  commentary,  and  the  result¬ 
ing  discussion  by  the  conference  participants.  Each  volume  may  be  read 
independently . 


Pask,  G.,  and  Robinson,  M.  (Ed.)  Current  approaches  to  scientific  decision 
making  in  complex  systems  III.  Vol.  2:  Position  papers.  Research 
Note  80-11,  in  press  (January  1980) . 

For  a  discussion  of  this  report,  see  the  preceding  entry  on  this  page. 


Pask,  G.  Cognitive  mechanisms  and  behaviors  involved  in  other  than  institu¬ 
tional  learning  and  using  principles  of  decision.  Research  Note  80-4, 

February  1980.  (AD  A084  066) 

The  report  describes  studies  of  individual  and  team  decisionmaking  in 
complex  command,  control,  and  communication  systems  involving  man-machine 
and  computer  regulated  interaction.  The  most  recent  advance  is  a  comprehen¬ 
sive  team  decision  system  able  to  exteriorize  normally  hidden  conceptual 
operations  as  behavioral  records.  The  system  has  training,  testing,  and 
operational  applications.  It  incorporates  an  on-line,  novel  concept,  in¬ 
terrogation  system,  records  for  which  (together  with  logging  the  condition 
of  a  space  environment)  provide  indices  of  decision-making  style  and  per¬ 
formance  which  are  compared  with  data  from  (also  novel)  tests  for  learning 
style.  The  system  also  incorporates  a  computer  implemented  planning  and 
representation  method  as  well  as  an  environment  in  which  the  decision  process 
is  instituted  by  essential  bifurcations.  The  method  is  a  microbehavioral 
conversation,  based  on  conversation  theory,  from  the  conversation  laboratory 
studies. 

Summary  analyzed  results  of  earlier  parts  of  the  research  project  are 
presented,  but  the  recent  research  is  exemplified  by  detailed  records  since 
work  is  continuing  under  Grant  DAERO-79-G-0009.  It  should  be  possible  to 
establish  indices  of  decision  style,  performance,  and  an  index  of  individual 
and  team  behavior  under  more  or  less  stringent  conditions,  but  the  data  are 
bulky  and  not  easily  analyzed.  Among  the  anticipated  results  are  relations 
between  learning  and  conceptual  styles,  team  stability,  the  alternation  of 
roles,  predictions  regarding  performance  quality  and  responsibility,  and, 
finally,  the  realistic  prospect  of  a  powerful  decision  aid  to  be  used  either 
regularly  or  as  an  optimal  response  device  under  circumstances  when  the  par¬ 
ticipants  have  no  opportunity  to  make  thoughtful  and  wise  decisions. 
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Price*  H.  E. ,  Fiore llo,  M.,  Lowry*  J.  C.,  Smith,  M.  G. ,  and  Kidd*  J,  S. 
Department  of  Defense  and  service  requirements  for  human  factors  R&D 
in  the  military  system  acquisition  process.  Research  Note  80-23,  in 
press  (July  1980) . 

This  research  note  presents  a  thorough  analysis  of  Department  of  Defense 
and  service-level  policy  requirements  for  human  factors  in  the  military  sys¬ 
tem  acquisition  process.  It  is  intended  as  a  companion  document  to  ARI 
Technical  Report  476*  and  as  such  provides  detailed  indoctrination  into  both 
the  system  development  process  and  human  factors  R&D.  It  will  be  particu¬ 
larly  useful  to  those  who  require  a  greater  understanding  of  this  process. 

After  an  explanation  of  each  system  development  phase,  human  factors 
R&D  requirements  are  integrated  into  the  system  development  framework.  This 
is  accomplished  through  direct  referral  to  Department  of  Defense  Directives, 
Specifications,  and  Standards.  Because  differences  occur  in  Army,  Navy,  and 
Air  Force  implementation  of  human  factors  R&D,  formal  service  documentation 
is  presented  for  human  factors  in  each  phase  of  system  development.  These 
documents  include  service  regulations  and  instructions.  Formal  service 
documentation  provides  only  that  human  factors  requirements  le  implemented, 
without  setting  forth  a  particular  implementation  plan.  Informal  service 
documentation  is  presented  to  illustrate  the  processes  by  which  human  factors 
R&D  is  implemented  in  system  development  programs  initiated  by  each  service. 


Price,  H.  E.,  Fiorello,  M. ,  Lowry,  J.  C.,  Smith,  M.  G. ,  and  Kidd,  J.  S.  The 
contribution  of  human  factors  in  military  system  development.  Techni¬ 
cal  Report  476,  in  press  (July  1980) . 

This  document  reports  on  an  effort  to  determine  (a)  the  conceptual 
basis  for  human  factors  contributions  to  military  system  development  and 
(b)  the  feasibility  of  an  impact  assessment  methodology  to  measure  the  value 
of  human  factors  R&D  in  system  development.  For  these  purposes,  two  parallel 
analytic  processes  were  implemented. 

One  analytic  process  provided  the  conceptual  basis  for  human  factors 
in  military  system  development.  First,  a  rationale  for  human  factors  con¬ 
tributions  and  products  was  prepared.  This  rationale  was  further  supported 
by  policy  documentation  containing  requirements  for  human  factors  R&D.  The 
second  analytic  process  provided  the  basis  for  evaluating  human  factors  con¬ 
tributions.  A  review  of  cost-benefit  analysis  techniques  applicable  to 
human  factors  was  performed  together  with  a  derivation  of  measurement  metrics. 
These  efforts  resulted  in  a  framework  for  performance  of  impact  assessment 
and  a  determination  that  it  is  a  feasible  methodology  for  application  to 
evaluations  of  human  factors  contributions  to  military  system  levelopment. 

This  technical  report  provides  recommendations  for  further  refinement 
and  validation  of  impact  assessment  in  the  measurement  of  the  value  of  human 
factors  efforts  and  products.  Recommendations  for  developing  human  factors 
impact  assessment  handbooks  are  also  provided. 
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Badre,  A.  N.  Rapraaenting  and  displaying  information  for  tactical  decision 

processing.  Technical  Report  495,  in  press  (November  1980) . 

One  of  the  main  tasks  required  in  developing  user-compatible  systems  is 
the  specification  of  criteria  for  the  development  of  algorithms  that  search 
for,  classify,  and  order  the  display  of  chunks  of  tactical  information  such 
that ‘they  are  meaningful  to  the  decisionmaker.  The  aim  of  this  study  was 
two-foldi  (a)  to  investigate  the  effects  of  the  sequential  displaying  of 
chunks  on  the  assimilation  of  tactical  information)  and  (b)  to  determine  the 
effects  of  presenting  a  tactical  scenario  as  a  member  of  a  sequence  of 
scenarios  on  chunking.- 

Two  experiments  were  designed  and  conducted.  The  general  procedure  was 
to  utilise  battlefield  scenarios- to  be  reconstructed  and  copied  under  vary¬ 
ing  conditions  of  presentation  and  viewing  by  a  select  group  of  battlefield 
commanders  (participants) .  The  results  indicated  that  presenting  information 
sequentially  and  incrementally  by  meaningful  chunks  leads  to  higher  recall 
accuracy  than  if  the  same  information  is  presented  either  in  nonmeaningful 
chunks  or  in  meaningful  chunks  that  are  viewed  in  an  order  different  from 
that  of  the  original  reconstruction. 

The  findings  show  no  difference  in  accuracy  of  recall  under  varying 
contextual  conditions,  suggesting  that  the  viewer's  recall  strategy  is  to 
foreground  the  entire  sequence  of  scenarios,  not  only  the  one  to  be  con¬ 
structed.  Sequential  context  had  no  effect  on  chunk  size.  The  availability 
of  an  invariance  in  the  sequence  of  scenarios  had  a  significant  effect  on 
the  order  in  which  symbols  were  reconstructed  and  on  chunk  content. 
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Atwood,  M.  E. ,  Turner,  A.  A.,  Ramsey,  H.  R. ,  and  Hooper,  J.  N.  An  explora¬ 
tory  study  of  the  cognitive  structures  underlying  the  comprehension  of 
software  design  problems.  Technical  Report  392 ,  July  1978. 

(AD  A073  727) 

An  experiment  was  conducted  to  evaluate  a  framework  for  the  study  of 
software  complexity  and  comprehension.  Basic  to  this  framework  is  the  con¬ 
cept  that  a  person's  knowledge  of,  and  experience  with,  software  design  af¬ 
fects  that  person's  ability  to  comprehend  a  software  problem  and  its  poten¬ 
tial  solutions.  Past  research  on  software  complexity  and  comprehensibility 
has  largely  been  based  on  the  assumption  that  complexity  is  a  function  of 
surface  properties,  such  as  variable  names  and  flow  of  control.  Such  mea¬ 
sures,  however,  ignore  the  effects  of  experience. 

Research  on  expert-novice  differences  in  problem  solving  suggests  that 
experts  possess  a  large  number  of  previously  developed  knowledge  structures, 
or  schemata,  that  can  be  used  to  understand  or  solve  the  current  problem. 
Research  on  text  comprehension  provides  theoretics]  concepts  and  experimental 
paradigms  that  are  useful  in  determining  the  structure  and  content  of  these 
experience-related  schemata. 

An  experiment  examined  the  knowledge  structures  used  by  participants, 
at  differing  levels  of  experience,  in  comprehending  software  system  speci¬ 
fications.  Six  participants,  at  each  of  five  levels,  studied  a  software 
system  specification  and  then  summarized  both  the  presented  specification 
and  the  probable  form  of  the  corresponding  software  design .  The  results 
indicate  that  software  designers  use  previously  learned  schemata  in  under¬ 
standing  a  software  design  problem  and  in  actually  constructing  a  design 
and  that  these  schemata  differ  as  a  function  of  experience.  In  addition, 
the  structure  and  content  of  these  schemata  were  investigated.  It  is  sug¬ 
gested  that  by  determining  the  structure  and  content  of  such  schemata,  soft¬ 
ware  complexity  and  comprehensibility  can  be  considered  in  a  more  meaningful 
manner. 


Atwood,  M.  E.,  and  Ramsey,  H.  R.  Cognitive  structures  in  the  comprehension 
and  memory  of  computer  programs!  An  investigation  of  computer  program 
debugging.  Technical  Report  TR-78-A21,  August  1978.  (AD  A060  522) 

A  theoretical  framework,  based  upon  recent  studies  in  cognitive  psy¬ 
chology  on  memory  for  text,  wa3  developed  to  explain  certain  aspects  of 
human  behavior  during  computer  program  comprehension  and  debugging.  A 
central  concept  of  this  framework  is  that  the  information  contained  in  a 
program  is  represented  in  a  programmer's  memory  as  a  connected,  partially 
ordered  list  (hierarchy)  of  "propositions"  (units  of  information  with 
properties  similar  to  those  observed  in  research  on  text  memory) .  An  ex¬ 
periment  was  performed  to  test  the  hypothesis  that  the  difficulty  in  find¬ 
ing  a  program  bug  is  a  function  of  the  bug's  location  in  this  hierarchy. 
This  experiment  compared  the  effects  of  bug  location,  bug  type  (array, 
iteration,  assignment)  and  specific  program.  Each  of  48  subjects  debugged 
two  separate  programs,  with  one  type  of  bug  at  two  different  hierarchical 
levels  in  each  program. 
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A  preliminary  analysis  suggested  that  all  three  factors — program,  bug 
type,  and  bug  location — significantly  affected  the  time  required  to  locate 
program  bugs.  Detailed  analyses,  however,  suggested  the  program  and  bug 
type  variables  could  be  explained  in  terms  of  the  bug  location  variable 
and  that  a  bug's  location  in  a  program's  underlying  propositional  hierarchy 
is  a  principal  factor  affecting  performance  in  a  comprehension  and  debugging 
task. 


Ramsey,  H.  R. ,  Atwood,  M.  E.,  and  Van  Doren,  J.  R.  A  comparative  study  of 
flowcharts  and  program  design  languages  for  the  detailed  procedural 
specification  of  computer  programs.  Technical  Report  TR-78-A22,  Sep¬ 
tember  1978.  (AD  A069  604) 

An  experiment  was  performed  to  assess  the  relative  merits  of  Program 
Design  Languages  (PDL's)  and  flowcharts  as  techniques  for  the  development 
and  documentation  of  detailed  designs  for  computer  programs. 

Twenty  students  in  a  computer  science  graduate  course  participated  in 
this  experiment.  Working  individually,  the  students  designed  a  two-pass 
assembler  for  a  simple  minicomputer.  Half  the  students  expressed  their  de¬ 
sign  for  the  first  pass  of  the  assembler  in  the  form  of  a  flowchart,  and 
expressed  their  design  for  the  second  pass  in  a  Program  Design  Language. 

The.  other  half  of  the  students  used  a  PDL  for  pass  one ,  and  a  flowchart , for 
pass  two.  Flowcharts  and  PDL's  were  compared  on  the  basis  of  various  mea¬ 
sures  of  overall  design  quality,  design  errors,  level  of  detail  of  designs, 
time  expended  in  developing  designs,  and  subjective  preferences. 

Having  completed  this  design  task,  the  subjects  then  performed  an  im¬ 
plementation  task.  They  were  given  fairly  detailed  procedural  designs  for  a 
program  which  simulates  the  function  of  a  fairly  sophisticated  minicomputer. 
They  were  then  required  to  develop  a  working  version  of  the  program  in  PL/1. 
Although  the  designs  were  logically  equivalent,  half  the  students  received 
their  simulator  design  in  flowchart  form,  and  half  in  PDL  form.  Flowcharts 
and  PDL's  were  compared  on  the  basis  of  design  comprehension  test  perfor¬ 
mance,  various  measures  of  overall  implementation  quality,  implementation 
errors,  and  subjective  preferences. 

In  the  context  in  which  this  study  was  performed,  the  use  of  a  Program 
Design  Language  (PDL)  by  a  software  designer ,  for  the  development  and  de¬ 
scription  of  a  detailed  program  design,  produced  better  results  than  did  the 
use  of  flowcharts.  Specifically,  the  designs  appeared  to  be  of  significantly 
better  quality,  involving  more  algorithmic  or  procedural  detail,  them  those 
produced  using  flowcharts.  In  addition,  flowchart  designs  exhibited  consid¬ 
erably  more  abbreviation  and  other  space-saving  practices  than  did  PDL  de¬ 
signs,  with  a  possible  adverse  effect  on  their  readability. 

When  equivalent,  highly  readable  designs  were  presented  to  subjects  in 
both  PDL  and  flowchart  form,  no  pattern  of  short-term  or  long-term  differences 
in  comprehension  of  the  design  was  observed.  No  significant  differences  were 
detected  in  the  quality  or  other  properties  of  programs  written  as  implementa¬ 
tions  of  the  designs,  subjective  ratings  indicated  a  mild  preference  for 
PDL's. 
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Overall,  the  results  suggest  that  software  design  performance  and 
designer-programmer  communication  might  be  significantly  improved  by  the 
adoption  of  informal  Program  Design  Languages,  rather  than  flowcharts,  as 
a  standard  documentation  method  for  detailed  computer  program  designs. 


Atwood,  M.  E.,  Ramsey,  H.  R. ,  Hooper,  J.  N. ,  and  Kullas,  D.  A.  Annotated 
bibliography  on  human  factors  in  software  development.  Technical  Re¬ 
port  P-79-1,  June  1979.  (AD  A071  113) 

As  part  of  a  larger  ARI  effort  to  survey,  synthesize,  and  evaluate  the 
state  of  the  art  in  the  area  of  human  factors  as  applied  to  software  devel¬ 
opment,  a. fairly  extensive  literature  survey  was  conducted.  This  resulting 
bibliography  contains  citations  of  478  articles  or  reports  pertaining  to 
the  behavioral  aspects  of  software  design,  programing,  coding,  debugging, 
testing,  evaluation,  and  maintenance.  Most  citations  are  accompanied  by 
descriptive  abstracts,  and  all  are  indexed  by  author,  publication  source, 
institutional  affiliation,  and  subject.  To  help  the  user  unfamiliar  with 
the  area,  the  bibliography  contains  brief,  basic  reference  lists  in  the 
areas  of  software  engineering,  the  psychology  of  software  development,  the 
Structured  Programming  Series,  and  the  DoD  software  program.  Coverage  is 
exhaustive  through  1977  with  a  few  references  in  1978. 


Ramsey,  H.  R. ,  Atwood,  M.  E.,  and  Campbell,  G.  D.  An  analysis  of  software 

design  methodologies.  Technical  Report  401,  August  1979.  (AD  A081  320) 

Four  formal  software  design  methodologies  were  described  and  .riefly 
analyzed:  (a)  Structured  Design,  (b)  Jackson's  Methodology,  (c)  Integrated 

Software  Development  System  (Higher  Order  Software) ,  and  (d)  Warnier's 
"Logical  Construction  of  Programs."  Relative  strengths,  weaknesses,  and 
commonalities  among  the  methods  were  identified  and  human  factors  problem 
areas  were  analyzed. 

Several  major  human  factors  deficiencies  and  problems  were  identified. 
Formal  software  design  methods  differ  in  terms  of  (a)  applicability  to 
problems  of  different  types,  size,  or  complexity;  (b)  susceptibility  to 
design  errors;  and  (c)  constraints  and  limitations  imposed  on  the  software 
designer.  Various  methods  limit  the  designer's  ability  to  select  an  ap¬ 
propriate  problem  representation,  prevent  the  designer  from  utilizing  rele¬ 
vant  knowledge  and  experience,  or  impose  potentially  significant  information 
loads  on  the  designer.  Improvements  in  design  methodologies  require  a  better 
understanding  of  the  problem-solving  behavior  of  software  designers;  poten¬ 
tial  research  topics  in  this  area  were  identified. 
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